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EXECUTIVE  SUMMARY 


A  Study  of  air  traffic  control  (ATC)  complexity 
issues  associated  with  the  causes  of  operational  errors 
(OEs)  in  the  National  Airspace  System  was  con¬ 
ducted.  It  consisted  of  a  comprehensive  survey  of  the 
literature  on  OEs  and  a  detailed  analysis  of  OE  data 
from  the  years  1992  through  1995  from  Atlanta’s  Air 
Route  Traffic  Control  Center  (ARTCC).  Of  specific 
concern  was  the  influence  of  airspace  sectorization 
and  traffic  characteristics  on  OE  incidence.  The  Sys¬ 
tematic  Air  Traffic  Operations  Research  Initiative 
(SATORI)  system  and  other  methods  were  used  to 
collect  data  for  the  analysis. 

The  literature  review  was  conducted  on  the  rela¬ 
tionship  of  airspace  and  air  traffic  factors  to  OE 
occurrence.  Papers  were  analyzed  for  information  that 
might  help  identify  sector  characteristics  or  aircraft 
flow  patterns  (also  known  as  ATC  complexity  factors) 
that  could  lead  to  a  loss  of  separation  between  aircraft. 
It  was  found  that  very  little  research  has  directly 
addressed  the  issue  of  ATC  complexity  as  a  factor  in 
OE  occurrence.  A  possible  reason  cited  was  that  most 
prior  investigation  has  focused  on  the  analysis  of  OE 
database  information  that  is  lacking  in  specific  details 
on  causality.  Nevertheless,  there  was  evidence  that 
sector  and  traffic  characteristics  could  influence 
workload  and  error  incidence. 

A  theme  in  the  literature  emerged  with  regard  to  the 
relationship  of  sector  design  and  the  amount  of  coor¬ 
dination  controllers  must  perform.  Many  studies  cited 
coordination  problems  as  a  factor  in  OEs.  Additional 
issues  from  the  literature  review  were  evaluated  for 
their  role  in  creating  the  conditions  for  OEs.  These 
data  came  from  both  theoretical  and  empirical  re¬ 
search  studies.  The  information  extracted  from  the 
literature  review  was  used  to  guide  the  research  at  the 
Atlanta  ARTCC. 

Data  from  SATORI,  a  complexity  factor  question¬ 
naire,  the  1995  facility  review,  and  OE  reports  were 
combined  in  a  database.  Descriptive  information  was 
computed  for  error  causality  and  sector  characteris¬ 


tics.  A  factor  analysis  of  the  sector  variables  yielded  six 
dimensions,  the  most  important  being  the  first  three: 
Traffic  Activity,  Size,  and  Military.  OE  conditions 
were  described  as  a  function  of  time  of  day,  flight 
level,  traffic  density,  complexity,  vertical  and  hori¬ 
zontal  separation,  severity,  workload,  number  of  con¬ 
trollers  working,  and  situation  awareness  (SA). 

The  45  sectors  in  the  Atlanta  ARTCC  were  catego¬ 
rized  into  No-,  Low-,  and  High-Error  groups.  The 
data  collected  on  sector  and  traffic  flow  characteristics 
were  used  to  explore  differences  between  groups  that 
might  account  for  OE  incidence.  Fifteen  variables 
had  statistically  significant  (or  marginally  significant) 
correlations  with  error  frequency.  Further  analyses 
indicated  that  four  variables  (Weather,  Radio  Fre¬ 
quency  Congestion,  Total  Complexity,  and  Average 
Complexity)  were  higher  for  the  High-error  group  as 
compared  to  the  Low-error  group.  Sector  size  was 
smaller  for  the  High-error  group,  as  compared  to  the 
combined  No-  and  Low-error  categories.  Four  other 
variables  showed  marginally  significant  differences 
between  groups. 

Using  several  related  statistical  techniques,  it  was 
possible  to  demonstrate  that  overall  OE  rate  or  sector 
severity  group  could  be  predicted  by  a  subset  of  the 
sector  characteristic  and  traffic  flow  variables.  How¬ 
ever,  although  there  was  statistical  significance,  accu¬ 
racies  were  generally  low,  indicating  insufficient  power 
for  practical  applications.  Nevertheless,  a  firm  theo¬ 
retical  relationship  was  demonstrated  between  sector 
complexity  and  OE  occurrence. 

The  data  were  also  analyzed  to  define  relationships 
between  sector  and  traffic  flow  characteristics  and 
controller  situational  awareness  at  the  time  of  the 
error.  The  only  statistically  significant  variable  was 
horizontal  separation;  more  horizontal  separation  was 
found  for  errors  where  SA  was  present.  Errors  where 
controllers  reported  that  they  were  not  aware  of  the 
impending  loss  of  separation  occurred  when  there 
were  more  military  operational  restrictions.  Also,  as 


sector  error  count  increased,  there  was  a  greater  pro¬ 
portion  of  errors  were  the  controller  did  not  have  SA. 
Thus,  high-error  rate  sectors  were  characterized  by 
low  SA  for  error  development. 

The  results  of  the  data  analysis  were  reviewed  for 
relationships  to  the  literature  survey.  Of  note  was  the 
24%  increase  in  error  causation  assigned  to  problems 
with  the  radar  display.  Previous  research  found  that 
most  errors  occurred  in  low  to  moderate  workload. 
The  current  data  set  showed  that  errors  were  reported 
with  an  average  of  eight  aircraft  in  the  sector  and  with 
moderate  complexity.  This  density  level  was  signifi¬ 
cantly  higher  than  the  base  rate  of  6.5  aircraft.  It  was 
not  possible  to  explore  the  issue  of  coordination 
problems  extensively.  There  was  some  indication  that 
sectors  with  climbing  and  descending  traffic  experienced 
more  OEs,  thus  corroborating  some  previous  findings. 


It  was  concluded  that  further  data  on  normal  At¬ 
lanta  ARTCC  traffic  flows  are  needed  to  determine  if 
OE  frequencies  departed  from  expected  proportions. 
The  factor  analysis  produced  useful  information  about 
the  underlying  dimensions  (Traffic  Activity,  Mili¬ 
tary,  and  Size)  characterizing  sectors.  One  facility 
review  variable,  Average  Density,  was  not  related  to 
other  traffic  load  measures.  This  warrants  further 
study.  Several  results  demonstrated  a  relationship 
between  sector  complexity  and  OE  rate.  Such  find¬ 
ings,  when  validated,  could  assist  with  traffic  manage¬ 
ment  and  sector  design  activities. 

Given  the  importance  of  situation  awareness  for 
avoiding  operational  errors,  evidence  was  found  that 
increased  sector  complexity  may  be  associated  with 
reduced  situation  awareness  and  may  lead  to  a  larger 
number  of,  and  more  severe,  errors. 
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The  Relationship  of  Sector  Characteristics 
TO  Operational  Errors 


1.  purpose 

A  research  project  was  conducted  on  sector  design 
and  operational  errors  (OEs)  in  en  route  air  traffic 
control  (ATC)  sponsored  by  the  Federal  Aviation 
Administration's  (FAA’s)  Civil  Aeromedical  Institute 
(CAMI).  The  goal  of  the  study  was  to  employ  the 
Systematic  Air  Traffic  Operations  Research  Initiative 
(SATORI)  system  and  other  sources  of  data  on  en 
route  ATC  sectors  to  investigate  possible  underlying 
causes  of  OEs  in  the  Atlanta  Air  Route  Traffic  Con¬ 
trol  Center  (ARTCC). 

2.  INTRODUCTION 

In  the  history  of  the  FAA,  no  aircraft  have  collided 
while  under  positive  control  in  en  route  airspace. 
However,  aircraft  have  violated  prescribed  separation 
minima  and  approached  in  close  proximity.  This 
event  can  occur  as  a  result  of  either  a  pilot  deviation 
or  an  OE.  An  OE  takes  place  when  an  air  traffic 
controller  allows  less  than  applicable  minimum  sepa¬ 
ration  criteria  between  aircraft  (or  an  aircraft  and  an 
obstruction).  The  number  of  OEs  is  a  primary  index 
of  National  Airspace  System  (NAS)  safety. 

Standards  for  separation  minima  are  described  in 
the  ATC  Handbook  (FAA  Order  711 0.65,  and  supple¬ 
mental  instructions).  While  there  is  considerable  com¬ 
plexity  in  those  standards,  at  flight  levels  between 
29,000  feet  (ft)  and  45,000  ft,  Air  Traffic  Control 
Specialists  (ATCSs)  at  en  route  facilities  are  required 
to  maintain  either  2,000  ft  vertical  separation  or  5 
miles  (mi)  horizontal  separation  between  aircraft.  At 
flight  levels  below  29,000  ft  with  aircraft  under  in¬ 
strument  flight  rules,  ATCSs  are  required  to  maintain 
either  1000  ft  vertical  separation  or  5  mi  horizontal 
separation. 

Immediately  after  the  detection  of  an  OE,  a  de¬ 
tailed  investigation  is  conducted  in  an  attempt  to  fully 
describe  the  events  associated  with  the  error's  occur¬ 
rence.  This  includes  removing  the  ATCS(s)  from  the 


operating  position  and  obtaining  a  statement  from 
each  of  the  involved  personnel,  gathering  the  relevant 
data  (voice  and  computer  tapes),  and  reviewing  in 
detail  the  events  associated  with  the  error's  occur¬ 
rence.  At  the  Atlanta  ARTCC,  the  SATORI  system  is 
used  to  re-create  the  error  situation  in  a  format  much 
like  the  one  originally  displayed  to  the  ATCS  (Rodgers 
and  Duke,  1993).  SATORI  allows  for  a  more  accurate 
determination  of  the  factors  involved  in  the  incident. 
Once  the  OE  has  been  thoroughly  investigated,  an 
OE  Final  Report  is  filed.  This  report,  the  Final  Opera¬ 
tional  Error/Deviation  Report  (FAA  7210.3),  con¬ 
tains  detailed  information  about  each  error  obtained 
during  the  investigation  process. 

The  responsibility  for  the  occurrence  of  an  OE, 
except  in  cases  of  equipment  outage  or  deficient 
procedures,  ultimately  rests  with  the  air  traffic  con¬ 
troller  who  must  detect  and  resolve  potential  conflicts 
before  they  become  errors.  However,  in  many  error 
analyses,  such  problems  are  blamed  on  some  type  of 
human  failure  without  a  deeper  investigation  of  other 
contributing  factors.  It  has  been  the  role  of  the  human 
factors  discipline  to  address  how  computer-human 
interface  (CHI)  characteristics  can  create  impedi¬ 
ments  to  task  performance,  resulting  in  errors.  This 
project  extends  this  approach  to  consider  characteris¬ 
tics  of  the  task  domain  or  environment  that  could 
increase  the  chances  of  error. 

The  discussion  of  airspace  and  traffic  characteris¬ 
tics  in  this  paper  is  based  on  the  concept  of  ATC 
complexity  (Mogford,  Guttman,  Morrow,  & 
Kopardekar,  1995).  This  construct  incorporates  the 
physical  aspects  of  a  sector,  such  as  size  or  airway 
configuration,  and  factors  relating  to  the  movement 
of  air  traffic  through  the  airspace,  such  as  number  of 
climbing  and  descending  flights.  Past  research  has 
identified  specific  airspace  and  traffic  factors  that 
contribute  to  ATC  complexity  (Mogford,  Murphy, 
Yastrop,  Guttman,  &  Roske-Hofstrand,  1993). 
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As  shown  in  Figure  1,  ATC  complexity  is  the 
underlying  driver  of  controller  workload.  The  proce¬ 
dures  required  in  the  sector,  flight  plans,  traffic  load, 
weather,  and  other  variables  form  the  basis  for  the 
tasks  the  controller  must  complete.  Although  evi¬ 
dence  is  weak,  controller  workload  is  probably  associ¬ 
ated  with  OE  commission.  As  task  information 
processing  requirements  reach  and  exceed  controller 
sensory  and  cognitive  capacities,  aircraft  may  not 
receive  sufficient  attention  and  control  to  maintain 
required  separation.  Workload  may  be  increased 
through  the  presence  and  interaction  of  several  com¬ 
plexity  factors  that  create  competition  for  similar 
cognitive  resources.  Alternately,  isolated  ATC  com¬ 
plexity  factors  may  lead  to  unsafe  conditions  by  plac¬ 
ing  focused  demands  on  the  controller.  Such  factors 
may  be  transitory  or  sustained  and  may  pose  undue 
strain  on  specific  information  processing  channels  or 
capabilities  (such  as  memory).  For  example,  the  manage¬ 
ment  of  a  sector  may  require  the  application  of  many 
required  procedures.  Forgetting  to  apply  these  at  the 
correct  time  could  lead  to  traffic  problems,  and  errors. 

The  amount  of  workload  experienced  by  the  con¬ 
troller  is  affected  by  the  information  processing  strat¬ 
egies  adopted  to  accomplish  required  tasks.  Such 
techniques  may  have  been  learned  in  developmental 
training  or  evolved  on  the  job  and  may  vary  in  effec¬ 
tiveness.  The  influence  of  a  complex  ATC  environ¬ 
ment  on  workload  can  be  ameliorated  through  the  use 
of  strategies  that  maintain  safety  through,  for  ex¬ 
ample,  simpler  or  more  precise  actions. 

Also  relevant  is  the  effect  of  equipment  on  workload. 
The  controller’s  job  will  be  made  easier  if  a  good  user 
interface  is  provided.  This  will  ensure  that  adequate 
and  accurate  information  is  presented  to  support 
efficient  task  completion.  Automation  tools  to  sup¬ 
port  essential  tasks  should  also  be  available.  It  is  for 
this  reason  that  the  FAA  is  developing  decision  sup¬ 
port  systems  to  facilitate  more  effective  control  ac¬ 
tions. 

Workload  can  also  be  influenced  by  personal  vari¬ 
ables,  such  as  age,  susceptibility  to  anxiety,  and  amount 
of  experience.  Variations  in  skill  between  controllers 
can  be  quite  pronounced.  These  factors  can  have  a 


strong  effect  on  the  workload  experienced  by  a  given 
controller  in  response  to  a  specific  array  of  ATC 
complexity  factors. 

The  goal  of  this  research  was  to  isolate  those  ATC 
complexity  factors  that  create  the  conditions  for  OEs, 
This  is  not  to  minimize  the  possible  effects  of  other 
CHI  and  operator-specific  factors.  Rather,  it  is  to 
begin  to  fill  a  research  void  on  ATC  errors  concerning 
the  effects  of  the  controller’s  work  domain:  air  traffic, 
airspace,  and  their  characteristics. 

The  paper  is  divided  into  two  sections.  The  first 
contains  a  brief  literature  review  on  OEs  and  their 
causes.  The  second  section  discusses  an  analysis  of 
data  available  on  OEs  from  the  Atlanta  ARTCC  and 
their  relationship  to  sector  and  traffic  characteristics. 

The  research  was  exploratory  in  nature  and  con¬ 
sisted  of  a  variety  of  analyses  intended  to  map  rela¬ 
tionships  in  the  data.  Exploratory  data  analysis  focuses 
on  generating  theoretical  models  from  data,  rather 
than  testing  a  pre-existing  model.  By  exploring  the 
data  with  an  inquisitive  mind,  it  is  possible  to  discover 
what  is  not  necessarily  expected.  Visual  representa¬ 
tions  (such  as  graphs)  and  the  use  of  statistics  that  are 
resistant  to  the  effects  of  departures  from  assumptions 
(such  that  variables  are  normally  distributed)  are 
used.  However,  one  outcome  of  using  several  statisti¬ 
cal  tests  with  the  same  data  set  is  an  increased  chance 
of  finding  statistically  significant  differences  where 
there  are  none.  As  a  result,  the  statistical  findings  in 
this  report  should  be  considered  as  tentative  pending 
a  larger-scale  study. 

3.  LITERATURE  REVIEW 

3.1  Scope 

This  review  examines  the  research  completed  to 
date  on  the  causes  of  OEs.  Literature  sources  included 
the  FAA  Technical  Center  Library,  FAA  Headquar¬ 
ters  Library,  FAA  CAMI  Library,  PsychINFO  (an  on¬ 
line  database  service  of  CompuServe),  National 
Technical  Information  Service  database,  and  Embry- 
Riddle  Aeronautical  University  Library.  Keyword 
searches  were  conducted  to  identify  publications  con¬ 
cerning  ATC  operational  errors  and  sector  design. 
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The  emphasis  of  the  literature  review  was  on  air¬ 
space-  and  traffic-related  factors  that  contribute  to 
OE  occurrence.  Although  the  papers  discussed  con¬ 
tained  information  on  other  factors  that  affect  error 
frequency,  they  were  not  the  focus  of  this  effort.  As 
much  as  possible,  the  contents  of  each  reference  were 
screened  for  insights  on  how  sector  design,  traffic  flow 
patterns,  procedures  (such  as  those  found  in  Letters  of 
Agreement  [LOAs]),  and  other  airspace  factors  cause, 
relate  to,  or  affect  OE  incidence.  Other  research 
findings  or  hypotheses  were  only  reported  as  needed 
to  support  the  discussion  of  this  theme. 

3.2  Summary  of  the  Literature 

Several  of  the  reviewed  papers  discussed  theoreti¬ 
cally-derived  airspace  issues  that  could  be  related  to 
OEs.  For  example,  Arad  (1964a)  conducted  an  ana¬ 
lytical  and  empirical  study  of  workload  in  relation  to 
sector  design.  He  divided  controller  workload  into 
three  categories:  the  background  load  involved  in 
working  the  position,  independent  of  aircraft  activity; 
the  routine  load  of  controlling  a  “standard”  aircraft, 
irrespective  of  its  interactions  with  other  aircraft;  and 
the  airspace  load  imposed  by  the  natural  tendency  of 
uncontrolled  traffic,  in  a  free-flow  environment,  to 
converge  in  unsafe  ways,  thus  requiring  control  actions. 

Arad  (1964a)  developed  a  mathematical  expression 
that  he  suggested  would  account  for  the  number  of 
aircraft  conflicts.  The  variables  in  the  equation  in¬ 
cluded  rules  of  separation,  average  traffic  speed,  num¬ 
ber  of  aircraft  under  control,  sector  size,  and  flow 
organization.  The  last  term  was  not  clearly  defined, 
but  was  described  as  “a  number  that  quantifies  the 
flow  organization  and  numerically  relates  the  vari¬ 
ables...  to  the  actual  numerical  value  of  the  conflict 
rate...”  (p.  29). 

A  report  by  Arad  (1964b)  described  further  work 
on  the  analysis  of  the  above  load  factors  and  suggested 
that  sector  design  could  greatly  affect  the  routine  load 
imposed  on  the  controller.  For  example,  if  traffic  flow 
tends  to  be  north/south,  then  establishing  sector 
boundaries  parallel  with  flow  imposes  less  work  on 
controllers  than  if  they  are  established  east/west,  or 


perpendicular  to  the  traffic  flow.  Therefore,  a 
rectangular  sector  with  its  long  side  parallel  with  the 
direction  of  traffic  flow  is  most  efficient  if  traffic  tends 
to  flow  in  one  direction.  If  traffic  flow  is  more  random, 
a  circular-shaped  sector  is  more  effective  and  will  pre¬ 
sumably  result  in  fewer  OEs. 

Along  similar  lines  to  these  papers,  Siddiqee  (1973) 
attempted  to  develop  a  mathematical  model  for  pre¬ 
dicting  the  expected  duration  of  aircraft  conflicts  at 
air  route  intersections.  The  equations  are  not  relevant 
for  this  review,  but  one  of  the  author’s  assumptions  is 
noteworthy: 

...in  the  en  route  environment,  aircraft  fly  essentially 
level  at  certain  standard  altitudes.  The  standard  altitude 
increments  used  are  large  enough  to  ensure  adequate 
vertical  separation,  with  allowance  for  flight  technical, 
altimeter,  and  pilot  errors.  Thus,  in  en  route  environ¬ 
ments,  conflict  situations  arise  mainly  because  of  loss  of 
the  horizontal  separation  among  aircraft  flying  at  the 
same  altitudes,  (p.  59) 

Schmidt  (1976)  described  a  sector  workload  model 
intended  to  aid  in  the  design  and  evaluation  of  air¬ 
space.  The  author  defined  ATC  workload  as  “the 
frequency  of  occurrence  of  events  which  require  deci¬ 
sions  to  be  made  and  actions  to  be  taken  by  the 
controller  team,  and  the  time  required  to  accomplish 
the  tasks  associated  with  these  events”  (p.  531).  Event 
categories  included  potential  conflicts  between  aircraft 
at  air  route  intersections,  potential  aircraft-overtaking 
conflicts  along  air  routes,  and  routine  procedural  events. 

Schmidt  (1976)  noted  that  the  expected  frequency 
and  duration  of  crossing  and  overtaking  conflicts  can 
be  predicted  by  traffic  flow  rate,  aircraft  separation 
standards,  route  geometry,  and  aircraft  velocity.  Con¬ 
flicts  at  an  air  route  intersection  are  related  to  aircraft 
flow  rate,  velocity  along  each  route,  minimum  aircraft 
separation  requirements,  angle  of  intersection  be¬ 
tween  the  routes,  and  number  of  flight  levels.  The 
author  developed  equations  that  purportedly  would 
predict  conflict  event  frequency  and  added  that  the 
amount  of  transitioning  traffic  had  to  be  factored  into 
the  calculation. 
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Couluris  and  Schmidt  (1973)  noted  that  sector 
design  can  affect  controller  workload  and,  presum¬ 
ably,  the  potential  for  OE  occurrence.  Number  of 
handoffs,  coordination,  pointouts,  and  structuring 
and  bookkeeping  events: 

...result  from,  or  are  influenced  by,  the  existence  and 
design  (shape)  of  the  sectors.  The  additional  work 
created  can  be  thought  of  as  the  cost  of  sectorization. 
Although  they  are  still  related  to  traffic  and  route 
parameters,  they  can  be  varied.  For  example,  a  sector 
boundary  that  crosses  a  highly  traveled  route  will  create 
a  larger  work  load  (from  the  above  four  work-produci  ng 
event  types)  than  a  boundary  across  a  sparsely  traveled 
route,  (p.  657) 

This  research  suggests  that  sector  structure,  in 
terms  of  boundary  location  and  shape,  airway  con¬ 
figuration,  and  intersections,  can  affect  conflict  fre¬ 
quency.  Such  factors  as  traffic  flow  rate,  average 
velocity,  separation  requirements,  number  of  flight 
levels,  and  transitioning  traffic  can  also  influence 
outcomes. 

The  following  studies  sought  to  classify  air  traffic 
controller  errors  and  may  shed  some  light  on  possible 
sector-related  causes  of  OEs.  They  were  screened  for 
possible  factors  that  could  help  orient  the  SATORI- 
based  research  on  Atlanta  ARTCC  airspace. 

Empson  (1987)  applied  a  human  error  classifica¬ 
tion  system  to  military  air  traffic  controllers  in  the 
United  Kingdom  over  a  two-year  period.  The  subjects 
performed  two  distinct  roles,  that  of  radar  director 
and  radar  approach.  The  radar  director  controller 
gave  heading  and  altitude  instructions  to  aircraft,  kept 
them  separated,  and  sequenced  them  for  approaches 
to  the  airfield.  The  radar  approach  controller  was 
responsible  for  aircraft  within  a  30-milc  radius  of  the 
field  and  initially  identified  aircraft  inbound  for  land¬ 
ing.  This  person  also  handled  aircraft  departing  from 
the  field.  The  radar  director  was  responsible  for 
flights  nearing  the  field  while  the  approach  con¬ 
trollers  handled  aircraft  within  the  approach  area. 
Approach  controllers  handled  about  twice  as  much 
traffic  as  the  directors. 

Eight  controllers  (four  in  each  category)  were  ob¬ 
served  over  two,  2y2-hour  periods.  Atotalof  131  errors 
were  observed  and  categorized  as  discrimination  errors, 


program-based  (or  action-related)  errors,  and  errors 
relating  to  memory  functions.  Errors  made  by  direc¬ 
tors  were  higher  than  for  approach  controllers.  This 
was  in  spite  of  the  fact  that  directors  had  aircraft  on 
frequency  36  percent  of  the  time  compared  with  79 
percent  for  approach  controllers.  The  error  rate  for  both 
controller  types  increased  with  traffic  load. 

The  author  suggested  the  reason  for  these  results 
might  be  that  the  director's  job  was  more  difficult. 
The  director  typically  had  to  accept  fast  reconnais¬ 
sance  jets  that  were  low  on  fuel  and  sequence  them  for 
handoff  for  landing.  Handoffs  had  to  occur  at  a 
precise  heading  and  altitude.  The  approach  control¬ 
ler,  on  the  other  hand,  experienced  much  less  time 
pressure  and  dealt  with  aircraft  with  various  destina¬ 
tions.  In  a  related  paper  by  Langan-Fox  and  Empson 
(1985),  it  was  suggested  that  another  reason  for  the 
higher  workload  in  the  director  position  could  be  that 
the  director  tasks  were  force-paced.  That  is,  in  the  case 
of  the  approach  controller,  the  presentation  rate  of 
events  were  system  controlled  as  opposed  to  worker 
controlled  (self-paced).  Other  research  (Bertelson, 
Boone,  and  Renkin,  1965)  demonstrated  almost 
error-free  performance  for  workers  in  a  self-paced 
work  setting  but  showed  a  dramatic  increase  in 
error  rates  when  the  self-pace  rates  were  imposed 
upon  the  operators. 

This  research,  apart  from  being  a  useful  application 
of  a  human  error  taxonomy,  demonstrates  possible 
effects  ofairspacc structure,  procedural  demands,  task 
characteristics,  and  traffic  type  on  controller  errors. 
The  findings  support  the  assumption  that  an  analysis 
of  such  features  can  lead  to  a  better  understanding  of 
controller  error. 

In  contrast  to  these  observations  of  working  con¬ 
trollers,  the  following  papers  focused  on  the  analysis 
of  OE  databases. 

Kinney,  Spahn,  and  Amato  (1977)  conducted  a 
study  of  system  errors  as  recorded  in  the  US  System 
Effectiveness  Information  System  (SETS)  for  the  years 
1974,  1975,  and  1976.  The  data  for  en  route  centers 
showed  that,  of  the  564  errors  recorded  for  the  3  years, 
95%  of  direct  causes  were  attributed  to  attention, 
judgment,  or  communications.  The  same  categories 
accounted  for  71%  of  contributing  causes.  Other 
contributing  factors  included  stress  (0.2%),  equipment 
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(6%),  operations  management  (5%),  environment 
(such  as  receipt  of  erroneous  information  or  lack  of 
compliance  from  pilots,  user  equipment  failures,  or 
heavy  controller  workload)  (1%),  procedures  (2%), 
external  (9%),  and  no  code  (6%). 

Only  one  of  the  error  categories  (procedures)  re¬ 
lates  to  sector  factors.  This  was  subdivided  into  six 
sub-categories  according  to  whether  the  procedure 
was  inadequate,  too  complex,  impractical,  etc. 

Kinney,  et  al.  (1977)  also  examined  system  errors 
in  the  context  of  reported  controller  workload.  T  raffic 
volume  and  workload  complexity  at  the  time  of  occur¬ 
rence  of  the  error  were  subjectively  rated  as  light, 
moderate,  or  heavy.  As  Table  1  shows,  most  incidents 
occurred  under  conditions  of  light  or  moderate  vol¬ 
ume  and  moderate  workload  complexity  in  the  1974- 
76  SEIS  data. 

Other  findings  by  Kinney,  et  al.  (1977)  showed  no 
relationship  between  errors  and  previous  errors  by  the 
same  controller  or  between  errors  and  controller  age. 
There  was  an  indication  in  the  1976  data  that  control¬ 
lers  with  more  than  5  but  less  than  ten  years  of 
employment  with  the  FAA  experienced  more  prob¬ 
lems.  Controllers  with  less  than  24  months  of  work 
history  in  their  current  positions  were  slightly  more 
error  prone.  Finally,  of  those  errors  reported  in  en 
route  centers,  54%  involved  aircraft  in  level  flight, 
26%  were  climbing,  and  20%  were  descending.  These 
results  tend  to  support  (by  a  few  percentage  points) 
Siddiqee’s  (1973)  assumption  that  most  en  route 
conflicts  should  be  in  the  horizontal  dimension. 

Schroeder  (1982)  examined  the  causes  of  loss  of 
separation  and  reviewed  the  FAA’s  OE  database  (SEIS) 
for  the  years  1970  through  1980.  Errors  per  million 
operations  in  terminal  airspace  steadily  increased  dur¬ 
ing  this  period.  However,  the  average  error  rate  re¬ 
mained  stable  for  en  route  airspace.  A  detailed  analysis 
of  the  SEIS  data  for  1977  and  1978  was  conducted.  As 
found  by  Kinney,  et  al.  (1977),  most  errors  occurred 
under  light  or  moderate  workloads.  Schroeder  (1982) 
identified  an  apparent  shift  in  error  patterns  from 
1965  and  1966  when  most  errors  occurred  under 
moderate  or  heavy  workload  conditions. 

Schroeder  (1982)  noted,  “Thus,  while  traffic  vol¬ 
ume  and  the  complexity  of  the  airspace  system  have 
increased  significantly,  a  higher  percentage  of  errors 


involve  light  to  moderate  workloads”  (p.  261).  He 
also  observed  that  workload  levels  rated  by  the  con¬ 
trollers  did  not  necessarily  depend  entirely  on  traffic 
density.  “There  are  obviously  other  aspects  of  the 
situation  that  become  involved  in  the  determination 
of  this  workload  measure  other  than  traffic  volume 
alone”  (p.  261).  Perhaps  these  other  aspects  could  be 
traffic  flow  or  sector  characteristic  variables. 

Kinney,  et  al.  (1977)  had  found  that  most  errors 
were  attributed  to  attention,  judgment,  and  commu¬ 
nications.  However,  the  recording  system  changed  in 
1978,  and  different  error  categories  were  adopted. 
Most  OEs  in  the  1978  and  1979  data  were  blamed  on 
a  failure  of  the  controller  to  initiate  corrective  action. 
Some  errors  were  caused  by  omitting  to  coordinate 
with  other  controllers  as  aircraft  crossed  sector  bound¬ 
aries.  Schroeder  (1982)  noted:  “In  fact,  a  review  of 
error  records  from  1969  through  1980  indicates  that 
coordination  was  either  a  direct  or  contributing  factor 
in  27.3%  to  53.9%  of  the  errors  for  both  en  route  and 
terminal  airspace”  (p.  264).  It  may  be  that  the  control¬ 
lers  were  not  clear  about  the  current  sector  configura¬ 
tion  in  their  areas  or  that  sector  design  created  an 
unacceptable  coordination  demand.  Other  important 
factors  were  flight  data  and  communication  prob¬ 
lems.  Schroeder's  (1982)  research  suggests  that  as¬ 
pects  of  sector  design  may  affect  the  way  aircraft 
transition  between  sectors  and  create  the  conditions 
for  OE  occurrence. 

Two  studies  on  OEs  in  the  Canadian  ATC  system 
were  conducted  by  Stager  and  Hameluck  (1990)  and 
Stager,  Hameluck,  andjubis  (1989).  They  found  that 
operating  irregularities  occurred  under  low  to  moder¬ 
ate  workload  conditions  with  none  to  normal  com¬ 
plexity.  Primary  error  categories  included  attention, 
judgment,  and  communication.  Like  Kinney,  et  al. 
(1977),  Stager,  et  al.  (1989)  distinguished  between 
direct  and  contributing  causes  with  regard  to  control¬ 
ler  errors.  Direct  causes  “refer  to  direct  actions  or  the 
failure  to  act  on  the  part  of  the  controller  that  results 
inescapably  in  a  loss  of  separation  given  a  certain 
system  state”  (p.  44) .  Contributing  causes  “refer  to  the 
specific  states  of  the  controller  (i.e.,  fatigue,  distrac¬ 
tion,  attitudes,  excessive  workload,  and  procedural 
knowledge)  as  well  as  the  states  of  the  environment, 
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including  task  design”  (p.  44).  Given  these  definitions, 
the  focus  of  this  analysis  is  on  contributing  factors  related 
to  airspace  structure  and  traffic  flow  characteristics. 

Redding  (1992)  reviewed  FAA  OE  reports  and 
incident  statements  for  1989  and  determined  that  a 
failure  to  maintain  situation  awareness  (SA)  was  the 
likely  cause  of  most  errors.  Errors  typically  occurred 
in  moderate  complexity  traffic  conditions  with  eight 
aircraft  under  control.  Communication  and  coordi¬ 
nation  problems  accounted  for  the  greatest  propor¬ 
tion  of  errors.  The  misidentification  or  misuse  of 
radar  data  was  the  attributed  cause  of  37.6%  of  total 
errors.  The  author  recommended  that  specific  in¬ 
struction  and  practice  in  maintaining  SA  be  added  to 
the  current  ATC  training  program. 

Schroeder  and  Nye  (1993)  reviewed  the  FAA’s 
Operational  Error/Deviation  (OED)  database  for  the 
years  1985  through  1988  to  examine  the  relationships 
between  workload  (traffic  load  and  complexity)  and 
causal  factors.  As  was  found  in  other  studies  (Kinney, 
et  al.,  1977;  Stager  and  Hameluck,  1990;  Stager,  et 
al.,  1989),  most  OEs  occurred  under  average  or  lower 
traffic  complexity  conditions.  Schroeder  and  Nye 
(1993)  found  differences  in  ratings  of  complexity  of 
the  traffic  situation  under  which  the  OEs  occurred 
across  ARTCCs  but  found  it  difficult  to  determine 
whether  these  differences  had  any  influence  on  OE 
occurrence. 

Some  relationships  were  noted  among  error  catego¬ 
ries.  When  the  radar  display  factor  was  involved  in  the 
OE,  statistically  significant  correlations  were  found 
with  communication  and  coordination  factors,  sug¬ 
gesting  that  these  three  variables  may  be  associated 
with  OE  incidents.  Overall,  the  radar  display  was 
involved  in  56.8%  of  OEs.  Communication  and 
coordination  were  cited  29.7%  and  29.6%  of  the 
time,  respectively,  while  data  posting  and  relief  brief¬ 
ing  were  associated  with  20.4%  and  4.2%  of  errors, 
respectively.  The  finding  of  coordination  problems  in 
these  data  is  similar  to  Schroeder’s  ( 1 982)  earlier  work 
in  this  area. 

Rodgers  and  Nye  (1993)  conducted  a  study  to 
relate  the  severity  of  OEs  to  air  traffic  controller 
workload,  as  measured  by  the  number  of  aircraft  being 
worked  and  the  complexity  level  at  the  time  of  the 
incident.  A  specific  question  was  whether  more  severe 


errors  occurred  during  periods  of  high  workload. 
There  was  also  interest  in  the  reported  causes  associ¬ 
ated  with  errors  and  whether  aircraft  flight  profile  and 
altitude  could  be  involved.  Finally,  investigations 
were  conducted  to  determine  the  underlying  factors 
leading  to  severe  OEs. 

The  OED  database  for  the  years  1 988  through  part 
of  1 99 1  was  used  for  this  study.  Average  traffic  load  at 
the  time  of  an  OE  was  8.8  aircraft.  It  was  found  that 
neither  traffic  load  (workload)  nor  air  traffic  complex¬ 
ity  was  related  to  the  severity  of  OEs.  The  authors 
noted  that  most  OEs  occurred  when  at  least  one 
aircraft  was  in  level  flight  and  at  least  one  was  descend¬ 
ing  or  ascending.  However,  most  moderately  severe 
errors  occurred  when  both  of  the  concerned  aircraft 
were  in  level  flight.  This  latter  statement  echoes 
Siddiqee’s  (1973)  hypothesis  that  en  route  ATC  er¬ 
rors  should  result  from  a  loss  of  horizontal  separation 
between  aircraft  at  the  same  level. 

Moderately  severe  errors  were  most  likely  to  result 
at  flight  levels  less  than  or  equal  to  29,000  ft.  Hori¬ 
zontal,  but  not  vertical  separation,  varied  as  a  function 
of  error  severity.  Higher  horizontal  separation  was 
associated  with  errors  where  the  controller  had  aware¬ 
ness  of  the  problem.  There  were  no  salient  factors 
found  to  explain  the  1  5  severe  OEs  found  in  the  data 
analyzed. 

Most  of  the  databases  containing  reports  on  OEs 
do  not  include  an  underlying  analysis  of  the  factors 
causing  the  error.  For  example,  although  judgment 
problems  might  be  cited,  it  is  not  certain  if  this 
resulted  from  sector  design  problems,  a  poor  com¬ 
puter-human  interaction  (CHI),  poor  controller  train¬ 
ing,  or  other  issues.  Fowler  (1980)  reviewed  ATC 
problems  from  a  pilot’s  point  of  view  and  echoed  such 
concerns.  He  noted  that  the  National  Transportation 
Safety  Board  typically  stopped  short  of  fully  examin¬ 
ing  the  human  factors  problems  associated  with  avia¬ 
tion  accidents.  He  suggested  that  some  errors  were 
symptoms  of  an  underlying  system  weakness.  Some  of 
the  human  factors  issues  in  the  ATC  system  that  relate 
to  the  issue  of  ATC  complexity  included  controller 
failure  to  coordinate  handoffs,  noncompliance  with 
procedures,  noncompliance  with  LOAs,  use  of  inap¬ 
propriate  procedures,  and  failure  of  managers  to  ad¬ 
vise  controllers  of  procedure  changes.  Adverse  weather 
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conditions  may  result  in  increased  turbulence,  icing, 
and  storm  activity  that  effectively  reduces  the  amount 
of  airspace  available  for  flights  and  may  result  in 
runway  closings.  Fowler  (1980)  thought  that  such 
sector-related  factors  could  contribute  to  OEs  and 
accidents. 

The  following  two  studies  used  simulation-based 
experimentation  to  analyze  the  factors  affecting  con¬ 
troller  workload  and  performance.  They  provide  an¬ 
other  viewpoint  in  the  search  for  evidence  of  the 
influence  of  ATC  complexity  on  OEs. 

Buckley,  DeBaryshe,  Hitchner,  and  Kohn  (1983) 
performed  two  experiments  to  assess  the  feasibility  of 
using  dynamic  real-time  simulation  procedures  for 
testing  ATC  systems.  The  purpose  of  the  work  was  “to 
determine  the  quality  of  measurement  of  system  per¬ 
formance  and  statistical  treatment  that  is  possible  and 
appropriate  in  dynamic  simulation  of  air  traffic  con¬ 
trol  systems”  (p.  1).  The  studies  identified  the  impor¬ 
tant  basic  dimensions  for  measuring  ATC  functions 
in  real-time  dynamic  simulations.  Of  interest  to  the 
topic  of  OEs  is  that  the  authors  addressed  the  issue  of 
the  effect  of  sector  geometry  and  traffic  density  on 
various  controller  performance  measures. 

The  first  experiment  examined  the  effects  on  per¬ 
formance  of  two  en  route  sector  geometries  and  three 
traffic  levels  ranging  from  very  light  to  very  heavy. 
Data  were  collected  from  two,  1-hour  runs  for  each  of 
31  controllers.  The  results  of  this  experiment  led  the 
researchers  to  conduct  a  less  complex  experiment 
using  only  one  of  the  possible  six  combinations  of 
conditions  of  sector  and  geometry.  This  second  ex¬ 
periment  examined  the  effects  of  replication  and  pro¬ 
vided  a  sufficient  amount  of  data  to  enable  the 
completion  of  a  factor  analysis.  Twelve,  1-hour  runs 
were  conducted  using  the  same  sector  with  the  same 
traffic  level  for  each  of  39  controllers. 

One  of  the  outcomes  of  the  first  experiment  was 
that  there  was  a  statistically  significant  effect  of  sector 
geometry  and  traffic  density  on  almost  all  of  the  10 
performance  measures.  There  was  also  a  significant 
interaction  effect  between  geometry  and  density. 
Buckley,  et  al.  (1983)  suggested  that  “Sector  [geom¬ 
etry]  and  [traffic]  density  are,  as  expected,  important 
factors  in  determining  the  results  which  will  occur  in 


a  given  experiment,  but  they  interact  in  a  complex 
way.  The  nature  and  extent  of  this  interaction  depend 
upon  the  measures  involved”  (p.  73).  This  research 
provides  evidence  that  both  traffic  and  sector  factors 
may  interact  to  affect  controller  performance  and, 
presumably,  the  possibility  of  OE  events. 

Stein  (1985)  conducted  a  simulation  experiment  to 
determine  the  relationship  between  a  number  of  air¬ 
space  factors  and  controller  workload.  Workload  was 
measured  by  the  Air  Traffic  Workload  Input  Tech¬ 
nique  (ATWIT)  in  which  the  controller  pressed  1  of 
10  buttons  on  a  console  with  1  representing  low 
workload  and  10  representing  high  workload. 

Ten  air  traffic  controllers  participated  in  a  series  of 
one- hour  simulations.  Subjects  experienced  a  low, 
moderate,  or  high  task  load  as  defined  by  the  number 
of  aircraft  in  a  sector  and  the  clustering  of  aircraft  in 
a  small  amount  of  sector  airspace.  Controller  input  to 
ATWIT  was  performed  once  per  minute.  Stepwise 
regressions  were  done  using  ATWIT  scores  as  a  crite¬ 
rion  measure.  Four  variables  produced  a  multiple 
correlation  of  i?  =  .85  with  the  workload  measure. 
These  were  (in  order  of  entrance  into  the  stepwise 
multiple  regression  equation)  clustering  of  aircraft  in 
a  small  amount  of  sector  airspace,  number  of  hand- 
offs  outbound,  total  number  of  flights  handled,  and 
number  of  hand-offs  inbound. 

The  study  demonstrated  a  strong  relationship  be¬ 
tween  controller  workload  and  a  subset  of  airspace- 
and  traffic-related  variables.  In  addition,  controllers 
were  able  to  provide  real-time  workload  estimates 
using  the  ATWIT  without  any  noticeable  decrement 
in  performance.  Workload  was  best  predicted  through 
a  multivariate  combination  of  airspace  variables.  The 
factors  listed  were  used  to  guide  research  into  error 
causation  at  the  Atlanta  ARTCC. 

Grossberg  (1989)  and  Mogford,  et  al.  (1993)  con¬ 
ducted  research  to  investigate  the  factors  comprising 
ATC  complexity.  Grossberg  (1989)  found  a  statisti¬ 
cally  significant  relationship  between  sector  complex¬ 
ity,  as  defined  by  FAA  Order  7210.46,  and  the  rate  of 
OE  incidence  at  the  Chicago  ARTCC.  The  correla¬ 
tion  was  statistically  reliable,  but  low  in  magnitude. 
This  provided  an  impetus  for  obtaining  more  infor¬ 
mation  on  factors  that  affect  sector  complexity. 
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Ninety-seven  controllers  rated  the  degree  to  which 
12  factors  contributed  to  the  difficulty  or  complexity 
of  operations  in  their  particular  sector  or  area  of 
specialization.  The  complexity  factors  most  frequently 
cited  in  the  Chicago  ARTCC  included:  control  ad¬ 
justments  involved  in  merging  and  spacing  aircraft, 
climbing  and  descending  aircraft  flight  paths,  mixture  of 
aircraft  types,  frequent  coordination,  and  heavy  traffic. 
Sector-related  factors,  such  as  large  sector  airspace  and 
intersecting  flight  paths,  received  lower  ratings. 

Grossberg  combined  the  factors  with  the  four  high¬ 
est  ratings  to  form  a  complexity  index.  He  found  that 
this  index  was  correlated  with  the  number  of  OEs 
found  in  sectors  in  the  Chicago  ARTCC.  Data  were 
collected  for  21  months  in  1987  and  1988.  The 
complexity  index  was  highly  correlated  (r  =  .74)  with 
frequency  of  OEs.  Correlations  between  the  standard 
FAA  formula  and  the  same  OE  database  correlations 
were  not  as  high  (r  =  .44). 

Mogford,  et  al.  (1993)  conducted  a  study  to  exam¬ 
ine  the  cognitive  processes  associated  with  ATC. 
Controllers  from  the  five  specialization  areas  in  the 
Jacksonville  ARTCC  participated.  The  purpose  of  the 
research  was  to  identify  complexity  factors  and  compare 
the  use  of  direct  (questionnaire  and  interview)  versus 
indirect  (statistical)  methods  for  factor  identification. 

Direct  methods  included  asking  controllers  to  sug¬ 
gest  and  then  rate  complexity  factors  in  terms  of  how 
they  made  sectors  more  or  less  difficult  to  control. 
Indirect  methods  involved  having  controllers  make 
paired  comparisons  with  respect  to  complexity  be¬ 
tween  maps  of  sectors  in  five  specialization  areas. 
Multidimensional  scaling  (MDS)  was  used  to  formu¬ 
late  complexity  factors  by  determining  whether  the 
arrangement  of  sectors  along  each  MDS  axis  corre¬ 
sponded  to  the  increase  or  decrease  in  some  variable  or 
factor  related  to  complexity. 

Thirteen  of  the  19  total  complexity  factors  were 
produced  by  both  methods,  showing  a  close  corre¬ 
spondence  between  direct  and  indirect  techniques  for 
determining  ATC  complexity  factors.  The  19  vari¬ 
ables  were  regressed  over  an  overall  complexity  crite¬ 
rion  formed  by  ratings  of  five  Traffic  Management 
Unit  staff  members  who  were  familiar  with  all  sectors 
in  the  ARTCC.  The  factors  of  complex  aircraft 


routings,  spacing  and  sequencing  for  departures  and 
arrivals,  and  frequency  congestion  during  peak  peri¬ 
ods  formed  a  significant  multiple  correlation  {R=  .85) 
with  the  overall  complexity  criterion. 

After  further  analysis,  the  factor  definitions  were 
refined  and  some  redundancies  removed.  The  follow¬ 
ing  16  unique  ATC  complexity  factors  were  identi¬ 
fied: 

1.  Number  of  climbing  and  descending  aircraft. 

2.  Degree  of  aircraft  mix. 

3.  Number  of  intersecting  flight  paths. 

4.  Number  of  multiple  functions  controller  must  per¬ 
form. 

5.  Number  of  required  procedures  controller  must 
perform. 

6.  Number  of  military  flights. 

7.  Frequency  of  contacts  (coordination)  or  interface 
with  other  entities. 

8.  Extent  to  which  controller  is  affected  by  airline 
hubbing. 

9.  Extent  to  which  controller  is  affected  by  weather. 

10.  Number  of  complex  aircraft  routings. 

1 1.  Extent  to  which  controller  is  affected  by  restricted 
areas,  warning  areas,  and  military  operating  areas. 

12.  Size  of  sector  airspace. 

13.  Requirement  for  longitudinal  sequencing  and  spacing, 

14.  Adequacy  and  reliability  of  radio  and  radar  coverage. 

1  5.  Amount  of  radio  frequency  congestion. 

16.  Average  amount  of  traffic. 

Although  not  specifically  addressing  sector  design 
issues,  recent  work  by  Rodgers  and  Manning  (1995) 
incorporating  SATORl  measures  into  a  study  of  OE 
occurrence  at  Atlanta  ARTCC  is  relevant  for  this 
review.  Data  for  12  OEs  that  occurred  between  1992 
and  1994  were  analyzed  using  the  Performance  and 
Objective  Workload  Evaluation  Research  (POWER) 
system,  a  subroutine  of  SA7  ORl.  (These  are  a  subset 
of  the  data  that  were  the  focus  of  the  current  research 
project.)  POWER  supports  the  collection  of  a  variety 
of  air  traffic  and  sector  measures. 

Seventeen  minutes  of  data  were  collected  on 
each  incident,  SVi  minutes  preceding  the  error  and 
SVz  minutes  during  the  error  interval.  Multivariate 
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analyses  of  variance  (MANOVAs)  were  calculated  for 
the  measures  in  Table  2.  Significant  differences  be- 
tween  the  period  preceding  the  error  and  the  error 
interval  were  found  for  the  variables  in  Table  3. 

The  results  suggest  controllers  were  busier  during 
the  error  period  than  the  preceding  period.  The  data 
also  show  that,  although  there  was  greater  vertical 
separation  during  the  error  period,  aircraft  density 
may  have  increased,  as  indicated  by  increased  time 
within  the  criterion  distance. 

Given  these  results,  it  is  not  certain  how  ATC 
complexity  factors  may  have  been  operating  in  these 
incidents.  However,  the  findings  shown  in  Table  3 
(especially  with  regard  to  handoffs)  could  be  consis¬ 
tent  with  coordination  problems,  a  theme  echoed  by 
other  authors  in  this  literature  review. 

3,3  Discussion 

Given  the  preceding  review,  it  is  helpful  to  summa¬ 
rize  factors  from  the  literature  that  are  relevant  to  the 
relationship  between  OEs  and  sector  or  traffic  fea¬ 
tures,  as  found  in  Table  4. 

The  most  relevant  observation  to  make  about  the 
above  summary  is  how  little  research  has  focused 
directly  on  the  topic  of  interest:  the  relationship  of 
sector  characteristics  to  OE  occurrence.  The  only 
author  who  directly  addressed  this  was  Grossberg 
(1989).  He  found  that  some  sector  and  traffic  charac¬ 
teristics  (or  ATC  complexity  factors)  were  correlated 
with  errors.  These  included  control  adjustments  to 
merge  and  space  aircraft,  climbing  and  descending 
aircraft  flight  paths,  mixture  of  aircraft  types,  frequent 
coordination,  and  heavy  traffic.  Even  in  this  author’s 
report,  it  is  difficult  to  determine  exactly  which  fac¬ 
tors  were  found  to  be  correlated  with  OEs. 

One  reason  for  the  absence  of  work  on  this  topic 
may  be  due  to  the  tendency  for  error-reporting  sys¬ 
tems  to  classify  errors  at  a  high  level  without  providing 
for  an  analysis  of  causality.  A  very  comprehensive 
review  of  the  FAA  OE  database  by  Kinney,  et  al. 
(1977)  found  that  most  OEs  were  attributed  to  con¬ 
troller  attention,  judgment,  or  communication  prob¬ 
lems.  However,  it  was  not  feasible,  given  these  error 
categories,  to  determine  if  controller,  CHI,  or  sector 
design  problems  were  involved.  It  is  impossible  to 
conduct  a  deeper  analysis  of  such  data  after  the  fact; 


the  OE  reporting  system  used  at  the  time  of  the  error 
defines  the  bounds  of  the  information  available.  Other 
efforts  to  classify  OEs  by  Schroeder  (1982),  Stager 
and  Hameluck  (1990),  Stager,  et  al.  (1989),  Redding 
(1992),  Schroeder  and  Nye  (1993),  and  Rodgers  and 
Nye  (1993)  have  had  to  deal  with  similar  issues  in 
reviewing  OE  database  information. 

Without  additional  information  concerning  the 
percentage  of  time  ATCSs  spend  controlling  traffic 
under  various  complexity  or  workload  conditions,  it 
is  difficult  to  determine  the  primary  factors  associated 
with  these  outcomes.  Unfortunately,  without  norma¬ 
tive  data,  one  must  settle  for  a  description  of  the 
factors  associated  with  operational  irregularities.  Ad¬ 
ditionally,  the  reporting  process,  including  the  re¬ 
porting  reliability  of  the  investigators,  may  affect  the 
extent  to  which  these  relationships  can  be  determined. 

In  spite  of  these  limitations,  the  data  suggest  sonie 
possibilities  about  the  relationship  of  ATC  complex¬ 
ity  factors  and  OEs.  Kinney  etal.  (1977)  observed  that 
2%  of  the  contributing  factors  listed  in  the  1974-76 
SEIS  reports  were  listed  as  problems  with  procedures. 
Although  this  term  can  be  used  to  describe  a  variety  of 
activities,  it  can  be  interpreted  as  referring  to  sector- 
specific  actions  the  controller  is  supposed  to  take  with 
regard  to  air  traffic.  However,  only  a  low  percentage  of 
contributing  factors  was  associated  with  this  category. 

Another  suggestion  referencing  or  implying  the 
role  of  complexity  factors  is  found  in  Schroeder  ( 1982) , 
who  implied  that  factors  apart  from  traffic  volume 
must  contribute  to  controller  workload  and,  presum¬ 
ably,  to  OE  occurrence.  Redding  (1992)  and  Schroeder 
and  Nye  (1993)  found  that  coordination  and  misuse 
of  displayed  data  accounted  for  many  errors. 

Coordination  is  also  a  theme  among  other  re¬ 
searchers,  such  as  Arad,  et  al.  (1964),  Couluris  and 
Schmidt  (1973),  Stein  (1985),  Fowler  (1980), 
Mogford,  et  al.  (1993),  and  (indirectly)  Rodgers  and 
Manning  (1995).  There  may  be  a  strong  relationship 
between  OE  occurrence  and  the  amount  of  coordina¬ 
tion  required  between  sectors.  The  frequency  of  this 
activity  is  largely  determined  by  the  location  of  sector 
boundaries.  If  sector  or  facility  boundaries  are  placed 
near  an  intersection  or  area  of  heavy  traffic  flow, 
controller  workload  and  the  probability  of  OEs  could 
be  increased. 
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Several  authors  noted  the  likelihood  of  OEs  as  a 
function  of  phase  of  flight.  Siddiqee  (1973)  predicted 
that  most  OEs  would  occur  as  a  result  of  a  loss  of 
horizontal  separation  between  aircraft  flying  at  the 
same  altitudes.  In  partial  support  ofthis  claim,  Kinney, 
et  al.  (1977)  found  that  most  errors  in  SEIS  data 
happened  in  level  flight  and  Rodgers  and  Nye  (1993) 
observed  that  moderate  errors  for  1988-90  OEs  usu¬ 
ally  involved  aircraft  in  level  flight.  However,  Rodgers 
and  Nye  (1993)  discovered  that  most  errors  were 
between  an  aircraft  in  level  flight  and  one  that  was 
climbing  or  descending. 

All  of  the  studies  that  focused  on  a  review  of  error 
database  information  found  that  OEs  occurred  under 
moderate  or  low  workload  conditions.  Rodgers  and 
Nye  (1993)  suggested  that,  theoretically,  SA  might 
decrease  under  high  workload  conditions.  However, 
given  the  lower  number  of  high  workload  OEs,  it  may 
be  the  case  that  SA  is  enhanced  as  workload  builds,  but 
decreases  (perhaps  due  to  fatigue)  as  workload  subse¬ 
quently  diminishes  (as  also  mentioned  by  Rodgers 
and  Nye,  1993).  Low  workload  may  not  foster  good 
controller  awareness  due  to  the  marginal  cognitive 
arousal  level  required. 

Several  of  the  studies  reviewed  were  either  theoreti¬ 
cal  in  nature  (Arad,  1964;  Schmidt,  1976;  Couluris 
and  Schmidt,  1973),  were  experiments  designed  to 
investigate  controller  workload  (Stein,  1985)  or  per¬ 
formance  (Buckley,  et  al.  1983),  or  were  analytical 
(Fowler,  1980)  but  have  information  applicable  to 
our  topic.  Arad  (1964)  developed  mathematical  mod¬ 
els  to  predict  conflicts  and  used  rules  of  separation, 
average  traffic  speed,  number  of  aircraft  under  con¬ 
trol,  sector  size,  and  flow  organization  as  variables. 
Schmidt  (1976)  took  a  similar  approach  and  stated 
that  conflicts  could  be  accounted  for  by  flow  rate, 
separation  standards,  route  geometry,  aircraft  speed, 
aircraft  flow  rate,  angle  of  airway  intersection,  and 
number  of  flight  levels.  Couluris  and  Schmidt  (1973) 
suggested  that  controller  actions  (such  as  handoffs, 
coordination,  and  pointouts)  are  affected  by  the  exist¬ 
ence  and  shape  of  sectors.  Stein  (1985)  found  that 
controller  workload  was  related  to  clustering  of  air¬ 
craft  in  a  small  amount  of  airspace,  number  of  hand¬ 
offs  outbound/inbound,  and  total  number  of  flights 
handled.  Buckley,  etal.  (1983)  discovered  that  traffic 


density  and  sector  geometry  interacted  to  affect  con¬ 
troller  performance.  Fowler  (1980)  thought  that  ATC 
complexity  was  affected  by  LOAs  and  weather. 

These  variables  could  be  considered  as  ATC  com¬ 
plexity  factors  in  the  same  vein  as  those  suggested  by 
Grossberg  (1989)  and  Mogford,  et  al.  (1993).  When 
combined  together,  a  substantial  list  of  factors  is 
suggested  that  can  be  used  in  research  on  ATC  com¬ 
plexity,  controller  workload,  and  OE  occurrence. 
These  variables  were  used  as  reference  points,  or 
guides,  while  analyzing  the  characteristics  of  Atlanta 
ARTCC  sectors  where  OEs  occurred. 

It  is  worth  mentioning  the  work  by  Empson  (1987) 
and  Langan-Fox  and  Empson  (1985)  as  a  different 
approach  to  the  previously  discussed  studies.  Rather 
than  reviewing  historical  data,  constructing  math¬ 
ematical  models,  speculating  on  causes,  or  conduct¬ 
ing  experiments,  these  authors  collected  real  time  data 
by  observing  military  controllers.  They  suggested 
some  of  the  now  familiar  factors  that  could  affect 
controller  workload,  such  as  airspace  structure,  proce¬ 
dural  demands,  and  traffic  type.  However,  they  dis¬ 
cussed  another  interesting  variable  in  noting  that 
errors  might  occur  more  frequently  when  controllers 
cannot  control  their  pace  of  work.  It  may  be  worth 
gathering  data  on  whether  sector  task  load  in  OE 
sectors  is  largely  driven  by  external  events  or  can  be 
modified  or  time-sequenced  by  the  controller. 

Finally,  further  research  on  OE  occurrence  should 
attempt  to  explain  the  findings  of  Rodgers  and  Man¬ 
ning  (1995).  It  may  be  that  ATC  complexity  factors 
can  help  account  for  the  observed  increases  in  sector 
transit  time,  handoff acceptance  latency,  vertical  sepa¬ 
ration,  and  aircraft  density  and  the  decrease  in  handoff 
acceptance  rate  associated  with  operational  errors. 

3.4  Conclusions  from  the  Literature  Review 

Although  considerable  work  has  been  completed  to 
attempt  to  understand  the  causes  of  OEs  in  en  route 
airspace,  much  of  this  research  has  been  limited  to 
using  the  available  information  contained  in  error 
databases.  Unfortunately,  this  approach  restricts  the 
investigator  to  the  data  contained  in  the  records  and 
precludes  in-depth  study  of  the  perceptual,  cognitive, 
or  environmental  factors  originally  at  play.  The  devel¬ 
opment  of  SATORI  has  made  it  possible  not  only  to 
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recreate  OE  incidents,  but  also  to  collect  data  on 
controller  activities,  sector  characteristics,  and  traffic 
patterns  during  the  time  of  the  error.  It  is  anticipated 
that  these  tools  will  allow  a  deeper  analysis  of  the 
conditions  surrounding  OE  occurrence. 

The  purpose  of  this  literature  review  was  to  locate 
research  relevant  to  the  relationship  of  sector  charac¬ 
teristics  and  traffic  flow  to  OEs.  It  was  found  that 
little  work  had  been  done  to  directly  address  this  issue, 
perhaps  due  to  the  limitations  inherent  in  the  avail¬ 
able  data.  However,  there  were  several  studies  that 
made  a  case  for  the  effect  of  ATC  complexity  factors 
on  controller  workload  or  error  frequency.  In  addi¬ 
tion,  it  was  possible  to  generate  hypotheses  about  the 
types  of  errors  that  might  occur  and  factors  that  could 
contribute  to  increases  in  workload,  complexity,  and 
the  probability  of  OEs.  These  findings  were  applied 
while  investigating  the  error  data  available  from  the 
Atlanta  ARTCC. 

The  confirmation  that  some  of  the  variables  iden¬ 
tified  in  this  literature  search  are  related  to  OE  occur¬ 
rence  is  only  the  beginning  of  an  effort  to  identify  the 
relevant  human  performance  issues.  Hopefully,  fur¬ 
ther  research  will  permit  the  development  of  a  human 
error  model  that  is  founded  on  factual  information 
available  from  the  system.  This  model  should  contain 
hypotheses  about  how  these  variables  are  perceptually 
and  cognitively  processed  by  the  air  traffic  controller 
to  result  in  an  error-prone  situation. 

4.  ANALYSIS  OF  OE  DATA 

To  more  fully  investigate  the  relationship  of  sector 
characteristics  and  traffic  flow  to  OEs,  and  to  explore 
some  of  the  predictions  and  assumptions  identified  in 
the  literature  review,  sector  and  OE  data  from  Atlanta 
ARTCC  were  analyzed. 

Information  on  sector  characteristics  was  collected 
using  the  SATORI  OpenCreate  application,  a  ques¬ 
tionnaire  based  on  the  previous  ATC  complexity 
work  by  Mogford,  et  al.,  (1993),  and  1995  Atlanta 
ARTCC  facility  review  data.  A  set  of  OE  data  was 
compiled  from  Atlanta  ARTCC  OE  Reports.  The 
following  sections  specify  the  variables  collected  and 


discuss  the  results  of  analyses  of  these  data.  The  data 
were  divided  in  two  sets,  focusing  separately  on  sector 
and  OE  characteristics. 

Given  the  exploratory  nature  of  these  analyses, 
results  were  considered  significant  if^  <  ./.  (Prob¬ 
abilities  that  rounded  to  .  1  were  included.)  Frequency 
count  variables  were  considered  to  be  acceptable  for 
inclusion  in  parametric  tests  if  they  contained  suffi¬ 
cient  categories  and  could  be  assumed  to  be  normally 
distributed  in  the  population.  Measures  that  had  five 
or  less  categories  were  not  included  and,  if  it  was 
indicated,  were  analyzed  using  non-parametric  tests. 

4.1  Sector  Data 

Data  were  collected  on  the  characteristics  of  sectors 
at  the  Atlanta  ARTCC  and  were  obtained  from  three 
sources.  First,  OpenCreate  was  used  to  extract  details 
on  the  following  variables  for  each  sector: 

1.  Number  of  major  airports. 

2.  Percentage  of  volume  the  sector  occupies  of  a  cube 
or  other  regular  geometric  shape.  ^ 

3.  Number  of  shelves  in  the  airspace. 

4.  Total  cubic  volume. 

5.  Number  of  VORTACs. 

6.  Number  of  obstructions. 

7.  Number  of  intersections. 

8.  Miles  of  victor  routes. 

9.  Miles  of  jet  routes. 

10.  Miles  of  other  routes. 

During  a  visit  to  the  Atlanta  ARTCC,  further  data 
were  collected  on  the  characteristics  of  sectors  using  a 
questionnaire  containing  16  ATC  complexity  factors 
(16CF)  adapted  from  Mogford,  et  al.  (1993).  One 
Airspace  and  Procedures  Specialist  from  each  area  in 
the  facility  rated  all  of  the  sectors  in  his  or  her  area  on 
all  16  variables  (using  7-point  scales).  The  total  com¬ 
plexity  score  was  calculated  by  summing  the  16  factor 
ratings  for  a  particular  sector. 

In  addition,  the  results  of  the  1995  facility  review, 
consisting  of  average  complexity  and  density  ratings 
for  each  sector,  were  included.  The  facility  average 
complexity  estimate  is  calculated  by  facility  personnel 


’  Many  sectors  have  a  very  irregular,  three-dimensional  shape.  This  measure  helps  estimate  sector  shape  complexity  by  determining  how  close 
the  sector  comes  to  the  shape  of  a  regular  geometric  object,  such  as  a  cube. 
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during  the  sector  validation  conducted  each  year. 
This  assessment  involves  estimating  the  sector  com¬ 
plexity  using  a  formula  that  weights  various  ATC 
functions  (FAA  Order  7210.46).  These  are  the  func¬ 
tions  and  their  associated  weights  (in  parentheses): 
number  of  departures  (5),  number  of  arrivals  (4), 
number  of  radar  vectored  arrivals  (2),  number  of  en 
route  aircraft  requiring  control  actions  (4),  number  of 
en  route  aircraft  not  requiring  control  actions  (2), 
number  of  emergencies  (4),  number  of  special  flights 
(3),  and  number  of  required  coordinations  (1).  These 
eight  functions  are  evaluated,  weighted,  and  totaled  to 
derive  the  sector  complexity  workload  value.  The 
density  rating  is  calculated  by  averaging  traffic  vol¬ 
ume  in  each  sector  over  a  three  day  period,  represent¬ 
ing  an  Atlanta  ARTCC  90'’*  percentile  traffic  day. 

4.2  Operational  Error  Data 

Quality  Assurance  (QA)  personnel  at  each  facility 
are  responsible  for  gathering  data  and  completing  an 
OE  Report  in  accordance  with  FAA  Order  7210.3 
(Facility  Operations  and  Administration).  For  the 
purposes  of  this  study,  a  number  of  fields  from  103 
OE  reports  from  Atlanta  ARTCC  were  coded  and 
entered  into  a  data  file.  Those  OEs  where  more  than 
one  sector  was  involved  (13),  no  final  report  was 
available  (4),  or  the  error  was  attributed  to  an  equip¬ 
ment  failure  (1),  were  not  included  in  this  analysis. 
This  left  a  sample  of  85  OEs,  covering  a  three  year 
period  from  June  1 992  to  June  1 995.  For  each  error, 
the  following  variables  were  available: 

1.  Report  number. 

2.  Date. 

3.  Time. 

4.  Flight  level. 

5.  How  many  controllers  were  charged. 

6.  Causal  factors. 

7.  Type  of  sector  (ultra  high,  high,  or  low). 

8.  Sector  number. 

9.  Radar  or  non-radar  controller  charged. 

10.  Number  of  aircraft  in  the  sector. 


1 1.  Estimated  traffic  complexity.  ^ 

12.  Vertical  separation. 

13.  Horizontal  separation. 

14.  Number  of  controllers  working  the  sector. 

15.  Whether  training  was  in  progress. 

16.  Number  of  controllers  working  (including  trainees). 

17.  Whether  the  sector  was  combined  with  another 
sector. 

18.  Whether  any  positions  were  combined. 

19.  Wiiether  another  facility  was  involved. 

20.  Whether  the  controller  was  aware  that  the  error  was 
developing. 

4.3  Combined  Data  Set 

To  examine  sector  differences  that  might  have 
contributed  to  OE  incidence,  the  sector  and  OE  data 
were  combined.  The  corresponding  sector  data  were 
attached  to  each  line  of  OE  data  (given  that  each  OE 
occurred  in  a  specific  sector).  A  result  of  connecting 
sector  features  with  OEs  was  that  sector  data  were 
counted  more  than  once  in  some  analyses,  given  that 
more  than  one  error  occurred  in  many  sectors. 

4.4  Sector  Characteristics 

This  section  reviews  the  sector  data  to  provide  basic 
information  about  the  characteristics  of  the  Atlanta 
ARTCC  airspace.  Selected  descriptive  information 
on  general  characteristics  of  all  sectors  in  the  center  is 
included  to  provide  a  basis  for  the  investigation  of 
sector  factors  that  may  be  associated  with  OEs. 

A  histogram  of  total  errors  (1992  to  1995)  for  the 
45  sectors  in  the  facility  is  found  in  Figure  2.  This 
forms  a  positively  skewed  distribution  with  a  mean  of 
1.9  errors,  a  standard  deviation  of  2.1  errors,  and  a 
median  of  2.0  errors.  (Each  bar  in  the  histogram 
represents  the  count  of  errors  between  the  value  of  the 
preceding  categor}"  and  the  value  of  the  labeled  category.) 

Figure  3  shows  the  distribution  of  sectors  for  size  or 
volume  of  airspace.  The  average  volume  was  30,5 1 7. 1 
cubic  miles  (cu  mi)  with  a  standard  deviation  of 
51,242.6  cu  mi  and  a  median  of  11,314.0  cu  mi. 
There  was  a  wide  range  of  sector  sizes  with  the  smallest 


^  Three  measures  of  ATC  complexity  for  Atlanta  ARTCC  sectors  arc  used  in  this  report,  llicsc  include  the  sum  of  the  16CF  factors,  a 
complexity  calculation  from  the  1995  Annual  Review,  and  the  estimated  complexity  of  the  traffic  situation  at  the  time  of  an  OF'. 
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being  4,695  cu  mi  and  the  largest  at  212,370  cu  mi.  A 
large  proportion  of  sectors  were  between  5,000  and 
15,000  cu  mi. 

As  shown  in  Figure  4,  most  sectors  were  low  (0  ft  to 
flight  level  [FL]230)  or  high  (FL240  to  FL340).  Nine 
sectors  were  ultra  high,  or  FL350  and  above. 

Average  traffic  density  (as  measured  during  the 
1995  facility  survey)  was  6. 1  aircraft,  with  a  standard 
deviation  of  1.7  aircraft  and  a  median  of  6.1  aircraft. 
The  distribution  of  sectors  for  average  traffic  density 
is  shown  in  Figure  5. 

A  factor  analysis  using  sector  variables  was  completed 
to  explore  the  underlying  structure  of  sector  characteris¬ 
tics.^  The  principal  components  extraction  produced  six 
factors  that  accounted  for  76%  of  the  variance  in  the 
measures.  Table  5  shows  the  unrotated  factor  matrix 
which  was  more  easily  interpreted  than  the  varimax 
rotation.  (Variance  accounted  for  is  in  the  first  row  of 
data.  Highest  variable  loadings  are  in  bold  type.) 

Fifty-six  percent  of  the  variance  was  explained  by 
the  first  three  factors.  The  first  factor  appeared  to  be 
related  to  traffic  volume  and  activities  associated  with 
managing  aircraft.  The  annual  facility  review  mea¬ 
sure,  average  complexity,  was  loaded  on  this  factor. 
Average  complexity  is  based  on  number  of  departures, 
arrivals,  en  route  aircraft,  emergencies,  special  flights 
and  required  coordination.  Factor  1  also  had  loadings 
for  climbing/descending  traffic,  frequency  conges¬ 
tion,  traffic  volume,  and  other  traffic  management- 
related  factors.  Accordingly,  Factor  1  was  called 
“Traffic  Activity.” 

Factor  2  was  named  “Size”  in  that  it  had  loadings 
for  cubic  volume  of  airspace,  miles  of  airways,  and 
sector  size.  Both  the  objective  and  subjective  measures 
of  sector  size  were  related  to  this  factor.  The  presence 
of  a  negative  weighting  of  aircraft  mix  implied  that  a 
low  degree  of  mix  (indicating  a  predominance  of 
larger,  jet  aircraft)  was  associated  with  larger  (and 
higher)  sectors.^  Miles  of  airways  and  sector  shelving 
were  also  part  of  this  factor. 


Factor  3  was  concerned  with  military  traffic  activ¬ 
ity  and  airspace.  It  also  had  a  loading  for  adequacy  of 
radio  and  radar  coverage.  Although  this  third  variable 
did  not  seem  to  be  directly  related  to  military  func¬ 
tions,  it  may  have  been  conceptually  associated  with 
military  airspace.  Areas  that  have  poor  radio  and  radar 
coverage,  or  are  controlled  by  the  military,  are  rela¬ 
tively  inaccessible  to  FAA  ATC.  Factor  3  was  called 
“Military.” 

Factor  4  was  only  concerned  with  the  number  of 
VORTACs  and  intersections.  Factor  5  was  not  well- 
defined,  in  that  it  only  had  a  loading  for  percentage  of 
a  regular  shape  (such  as  a  cube)  that  the  sector  filled. 
Factor  6  had  the  distinction  of  being  associated  with 
average  density,  the  second  annual  review  measure. 
The  fact  that  this  variable  was  not  associated  with 
Factor  1  suggests  that  it  may  not  actually  be  measur¬ 
ing  traffic  activity  but  some  other  sector  characteris¬ 
tic.  There  were  two  variables,  average  density  and 
number  of  intersections,  that  were  identified  with 
more  than  one  factor. 

4.5  Operational  Error  Analysis 

General  OE  causality  information  was  extracted 
from  the  OE  Reports.  Then  the  sector  and  OE  data 
sets  were  analyzed  to  search  for  any  patterns  that 
might  help  explain  OE  occurrence. 

4.5.1  Operational  Error  Causal  Factors 

Each  of  the  85  OEs  was  categorized  by  primary  and 
secondary  causal  factors  at  the  time  of  the  error. 
Tables  6  and  7  show  the  breakdown  of  primary  and 
secondary  causes  by  number  and  by  sector.  (Only  22 
errors  were  assigned  secondary  causal  factors.) 

Many  of  the  errors  (a  total  of  70%)  were  primarily 
attributed  to  problems  interpreting  the  radar  display. 
Eighteen  percent  of  the  errors  were  due  to  communica¬ 
tion  problems  (including  transposition,  misunderstand¬ 
ing,  readback,  and  acknowledgment).  Coordination  and 
data  posting  accounted  for  5%  and  6%  of  the  OEs. 


^  The  multivariate  procedures  used  in  this  study  were  often  based  on  a  relatively  low  number  of  cases.  It  is  recognized  that  a  larger  data  set 
will  be  necessary  before  drawing  firm  conclusions  from  such  analyses.  A  correlation  table  for  these  variables  is  in  Appendix  A. 

'‘A  one-way  analysis  of  variance  determined  that  there  was  a  significant  difference  in  sector  size  as  a  function  of  altitude  level,  E(2, 44)  =  34.66, 
p  =  .000).  Tukey  Honestly  Significant  Difference  post  hoc  tests  showed  a  significant  difference  between  low  or  high  sectors  and  ultra  high 
sectors,  with  higher  sectors  being  larger. 
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Secondary  causal  factors  were  largely  attributed  to 
communication  errors  and  radar  display  problems. 
Table  8  shows  a  summary  of  all  reported  causal 
factors.^ 

4.5.2  Operational  Error  Conditions 

This  section  reviews  the  quantitative  OE  data, 
analyzing  the  circumstances  under  which  errors  oc¬ 
curred.  Figure  6  shows  the  distribution  of  OEs  by 
time  of  day.  It  appears  that  most  errors  occurred 
between  0800  and  2000  hours.  This  probably  corre¬ 
sponds  with  normal  traffic  flows  at  the  Atlanta 
ARTCC,  although  no  data  were  available  on  overall 
traffic  count  over  time. 

Figure  7  is  a  distribution  of  OEs  by  flight  level. 
There  were  spikes  in  the  OE  count  between  1 5,000  ft 
and  20,000  ft  and  again  between  FL300  and  FL350. 
Figure  8  is  another  rendition  of  OE  by  level,  except 
that  it  shows  the  distribution  of  errors  by  sector  type 
(low,  high,  and  ultra  high). 

Figure  9  is  a  combination  of  Figures  4  and  8, 
showing  the  proportion  of  sectors  and  OEs  by  sector 
type.  It  appears  that  a  disproportionately  high  number 
of  errors  occurred  in  high  sectors  while  fewer  than 
might  be  expected  were  recorded  in  ultra  high  sectors. 
A  chi  square  test  of  these  data  showed  that  there  were 
significantly  different  proportions  of  errors  than  would 
be  expected  given  the  number  of  sectors  in  each  group, 
5^2(85)- 12.43,/  =  .002.  However,  this  result  might  be 
related  to  normal  traffic  flow  patterns  in  each  sector  type. 

Traffic  count  data  at  the  time  of  each  error  are 
plotted  in  Figure  10.  The  distribution  of  errors  by 
number  of  aircraft  in  the  sector  when  separation  was 
lost  approximates  a  normal  curve  with  a  mean  of  8.0 
aircraft  and  a  standard  deviation  of  2.9  aircraft.  When 
comparing  these  data  with  the  average  density  of 
sector  traffic  reported  in  Figure  5,  it  is  evident  that 
traffic  density  at  the  time  of  an  error  was,  on  the 
average,  1.5  aircraft  higher.^  This  difference  was  sig¬ 
nificant  with  134.94)  =  4.12,  /  =  .000. 


Figure  1 1  is  a  plot  of  OE  count  by  estimated 
complexity  rating  (from  the  OE  report).  The  com¬ 
plexity  of  the  air  traffic  situation  at  the  time  of  the  OE 
was  assigned  after  the  error  occurred  on  a  scale  of  1 
(low)  to  5  (high).  Average  complexity  was  3.4  with  a 
standard  deviation  of  1.2  and  a  median  of  4.0.  Most 
errors  were  found  in  the  moderate  (3  or  4)  range,  OE 
report  complexity  had  low,  but  significant,  correla¬ 
tions  with  average  (annual  review)  sector  complexity 
(r  =  .24,/  =  .030)  and  total  16CF  ATC  complexity  (r 
=  .40,/  =  .000).  It  had  a  high  correlation  with  number 
of  aircraft  in  the  sector  at  the  time  of  the  error  (r  =  .86, 
/  =  .000)  and  low,  but  significant,  correlations  with 
16CF  traffic  volume  (r  =  .24,  /  =  .029)  and  average 
(annual  review)  sector  traffic  density  (r  =  .2 1 , /  =  .052). 

Figure  12  is  a  plot  of  OE  frequency  by  workload 
index.  Workload  was  calculated  based  on  an  approach 
developed  by  Rodgers  and  Nye  (1993).  The  index  is 
the  sum  of  the  z  scores  for  complexity  and  traffic 
count  at  the  time  of  the  error.  It  takes  into  account 
both  sector  and  traffic  factors  in  estimating  controller 
workload.  As  can  be  seen  in  Figure  12,  the  distribu¬ 
tion  is  somewhat  negatively  skewed,  indicating  that 
workload  for  many  errors  was  higher  than  the  average  for 
all  errors. 

Figure  13  is  a  histogram  of  OEs  by  amount  of 
vertical  separation.  The  errors  were  divided  into  two 
groups:  below  FL290  and  FL290  and  above,  given 
that  there  are  two  separation  standards  in  en  route 
airspace.  (Below  FL290,  1000  ft  is  required,  while  at 
FL290  and  above,  2000  ft  is  the  minimum.)  As  seen 
in  the  graph,  two  concentrations  of  separation  at 
closest  point  of  approach  were  found,  corresponding 
to  the  two  standards.  Although  there  were  some  errors 
with  less  than  1000  ft  of  separation  at  the  higher 
levels,  most  had  at  least  1000  ft. 

Figure  14  depicts  the  horizontal  separation  be¬ 
tween  aircraft  pairs  at  the  time  of  OEs.  The  distribu¬ 
tion  is  negatively  skewed  with  a  mean  of  3.6  mi,  a 
standard  deviation  of  0.9  mi,  and  a  median  of  3.8  mi. 
Most  errors  occurred  with  3  or  more  miles  of  horizon¬ 
tal  separation  remaining. 


^  The  percentage  total  sums  to  more  than  100  because  more  than  one  cause  may  have  been  attributed  to  each  error. 

®  The  mean  of  the  annual  review  average  density  for  all  sectors  was  6.5  aircraft  in  this  calculation.  The  average  traffic  density  for  each  sector 
associated  with  an  error  was  included,  resulting  in  some  sectors  being  counted  more  than  once  in  the  average.  This  resulted  in  a  different  mean 
than  that  computed  for  Figure  5. 
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The  hofiaonial  and  vertical  separation  data  were 
combined  into  one  separation  measure  by  calculated 
minimum  root  mean  square  (RMS)  distance  for  each 
OE.  Figure  1 5  shows  the  RMS  distance  for  OEs  that 
occurred  below  and  above  FL290,  The  mean  separa 
tion  was  21782.8  ft,  with  a  standard  deviation  of 
5641,8  ft  and  a  median  of  2284 1.1  ft 

Figure  1 6  shows  the  number  of  controllers  working 
at  the  time  of  the  error.  One  controller,  in  the  radar 
position,  alv/ays  works  the  sector  As  traffic  increases, 
a  second  contiullet  in  the  data  position  is  added. 
During  extremely  busy  periods,  an  assistatitcontiollei 
is  assigned.  As  can  be  seen  in  the  graph,  rtrost  errors 
occurred  with  one  or  two  conttolleis  working  How 
ever,  tliis  may  nicrely  reflect  normal  staffing  patterns 

One  of  the  items  contained  oii  the  final  report 
prepared  after  the  oecurtence  of  ari  OE  requites  an 
assessment  of  the  involved  employee's  awareness  of 
tire  develupittg  etrot  This  item  Iras  Leeii  on  tlie  OE 
final  reporting  form  for  the  past  1 4  After  listen 
ing  to  the  assoeiated  voice  tape.  Inlervievving  the 
involved  controller,  and  tcviewiiig  the  error  v,'ith 
SATORl,  quality  assurance  (QA)  specialists  make  a 
determitiatioii  as  to  the  controller's  awatetitss  Al 
though  SATOBI  simplifies  the  fotmulatioii  of  this 
judgment,  most  QA  specialists  fitid  the  answer  tela 
lively  easy  to  ascertain. 

Typically,  if  either  the  control  aotion  to  provide 
separation  was  not  issued  in  a  timely  manner,  or  no 
control  action  was  initiated,  the  conuollet  is  judged 
to  be  unav/aic  of  the  developing  error  I  lo  wever,  if  the 
controller  actively  attempted  to  provide  separation  to 
the  involved  aircraft,  although  the  control  action  was 
either  inappropriate  or  inadequate,  the  controller  is 
judged  to  be  aware  of  the  developing  error.  In  73%  of 
the  cases  in  the  OE  data  set,  the  controller  was  found 
to  be  unaware.  Further  analysis  of  OEs  with  regard  to 
controller  SA  is  found  in  Section  4.5,4. 


4.5.3  Accounting  for  Error  Frequency 

One  of  the  primary  goals  of  this  project  was  to 
determine  if  sector  or  traffic  characteristics  could 
account  for  OE  incidence.  Three  techniques  based  on 
the  general  lineat  model  were  used  to  explore  these 
relationships.  First,  MANOVA  was  applied  to  deter 
mine  if  diffetenccs  existed  between  sectors  with  no 
ettors,  few  errors,  or  ntarty  errors. 

In  Figure  2,  the  distribution  of  errors  in  the  4S 
sectors  in  the  Atlanta  ARTCC  was  examined,  Witli  a 
mean  of  1.9  errors  and  a  standard  deviation  of  21 
etiuts,  it  v/as  decided  to  separate  the  sectors  iiilo  0£ 
fiequetioy  gtor4ps  with  0  oiroi'S,  less  thari  4  ertois  (low 
etiot  sectors),  and  4  or  more  ertots  (high  error  sec 
tors).^  There  were  1 5, 22.  and  8  sectors  in  each  group, 
respectively. 

Bivariate  Fearson  and  Speatrrran  cottelations  were 
eaioulaied  bctv.'cen  29  variables  from  Section  4  ]  aitd 
the  Seetur  OE  frequency  niCaSUles  (rrumber  of  OEs 
arid  OE  frequettey  gtoirp)  All  curreiations  significant 
at  the p  <  are  listed  in  Table  9  ® 

Tire  Currelatioii  results  were  used  to  sereeit  vaii 
ableS  ftJl  ittchisiori  in  the  MAHOVA,  v.'ith  OE  fie 
queney  group  as  the  independent  variable  Eiglii 
vatiables  achieved  statistical  significance  of^  r.  .05  or 
less  These  were  included  in  the  MANOVA  which  was 
significant  with  Hoiellings  F(l6;  68)  rr,  1.60,  p 
.094®.  Four  dependerrt  variables  emetged  with  statis 
tically  significant  conuiLutioiis  {p  <  .05)  to  this  result 
Four  other  variables  weie  significant  at  the p  <  A  level. 

Tabic  1 0  showsone  way  analysis  of  variance  (AITOVA) 
tests  for  the  eight  OpcnCreate,  1 6CF,  artd  facility  tevievv 
data  variables  that  were  iiicludcd  in  the  MANOVA 
Results  wheie  p  <  .05  indicate  statistically  significant 
difFerences  in  sector  characteiistics  betweet,  errot  groups 
are  shown  in  bold  print,  (There  were  also  several  tests 
significant  at  the  p  ^  .1  level.)  Tukey  Honestly  Signifi 
cant  Difference  post  hoc  tests  were  performed  on  the 


^  The  division  beiwecn  lyw  and  higli  eiiwi  scviuis  Wits  sei  at  one  siandaid  deviauon  hum  the  mean,  oi  at  4,1)  euois 
‘^Results  atoriiearthe^=  4I  level  weref  reported  in  these  analyse*^  given  that  the  emphasis  on  dam  exploration.  In  thisa^^e,  a  lype  1  ttmi 
(incorreet  detection  or  a  difference)  would  only  lead  to  further  Investigation  with  a  larger  dam  m-  A  complete  Pearion  correlation  table  for 
the  sector  variables  in  Section  4.?»  is  found  In  Appendix  A,  A  few  of  the  variables  did  not  rneet  the  assumptions  of  normality  and 
homoseedasticity,  so  the  results  should  be  interpreted  with  caution 

^  Only  those  variables  correlated  with  p  <  .05  were  included  In  the  M  ANQ  VA:,  ']  \m  wat;  u?  conuol  thr  mnnbet  of  vaiiablcs  in  tlu:  anuly:u; 
and  to  improve  the  reliability  of  the  outcome 


significant  ANOVAs  (with/><  .05).  In  each  instance,  the 
only  significant  contrasts  were  between  the  zero  and  the 
high-error  sectors. 

Number  of  major  airports,  although  correlated 
with  OE  group,  was  not  include  in  the  MANOVA 
because  the  variable  contained  too  few  categories.  It 
was  not  possible  to  conduct  a  chi  square  test  against 
OE  category  because  of  zero  frequency  counts  in  some 
cells  of  the  contingency  table. 

Although  sector  size  did  not  emerge  as  a  discrimi¬ 
nator  for  OE  frequency  groups  in  this  analysis,  a  trend 
noted  in  the  data  is  worth  reporting.  Figure  17  shows 
a  graph  of  sector  size  as  a  function  of  OE  group.  The 
variance  evident  within  the  no  and  low-error  groups 
(as  represented  by  the  standard  deviation  error  bars) 
was  probably  the  reason  that  significant  differences 
were  difficult  to  detect  using  ANOVA.  However,  it  is 
clear  that  the  high-error  sectors  were  distinct  in  hav¬ 
ing  a  consistently  smaller  volume  than  the  zero  or  low- 
error  sectors.  This  is  supported  by  a  statistically 
significant  t  test  (in  spite  of  the  high  variability) 
between  the  combined  no  and  low-error  groups  versus 
the  high-error  group,  /(37)  =  2.57, =  OH. 

Another  analysis  of  differences  between  sectors 
with  low  and  high  OE  frequencies  was  conducted  to 
consider  relevant  OE  variables.  The  difference  be¬ 
tween  this  and  the  preceding  analysis  is  that  OE 
characteristics  recorded  at  the  time  of  the  error,  as 
opposed  to  general  sector  features,  were  being  consid¬ 
ered.  The  goal  was  to  determine  if  OEs  that  occurred 
in  sectors  with  many  errors  were  different  from  those 
taking  place  in  low  OE  sectors. 

Table  1 1  shows  the  bivariate  Pearson  and  Spearman 
correlations  (for  only  those  variables  where  /><.!) 
between  sector  OE  frequency  measures  (number  of 
OEs  and  OE  frequency  groups)  and  relevant  OE 
variables  from  Section  4.2. 

A  MANOVA  was  not  conducted,  in  that  only  one 
variable  had  a  significant  correlation  of <  .05  with 
OE  group.  However,  the  univariate  ANOVA  for 
number  of  aircraft  in  the  sector  was  significant  at  < 
.1,  with  means  larger  for  the  high-error  sector  group 
(M  =  7.49  versus  M  =  8.55).  The  ANOVA  conducted 
for  complexity  was  significant  at  y?  <  .05.  In  the  low- 
error  group,  the  mean  for  complexity  was  M  =  3.05 
and  in  the  high-error  group  it  was  M  -  3.64.  Also,  the 


test  involving  workload  index  was  significant  at  />  < 
.05.  with  means  larger  for  the  high-error  sector  group 
{M  -  -.44  versus  M  ~  .41). 

A  chi  square  test  for  position  combined  revealed  a 
significant  interaction  between  low  and  high-error  group 
and  whether  the  sector  was  combined  with  another 
sector  at  the  time  of  the  error,  X^(l)  =  7.13,/>  =  .008. 
Positions  were  combined  less  frequently  in  sectors  with 
a  high  frequenc}^  of  errors. 

Two  other  techniques,  multiple  regression  and 
discriminant  analysis,  were  employed  to  predict  OEs 
using  the  variables  from  Section  4. 1 .  The  relevant  and 
permissible  sector  characteristic  variables  identified 
in  Section  4.1  that  had  significant  correlations  with 
number  of  errors  were  submitted,  using  a  stepwise 
procedure,  into  a  multiple  regression  analysis  with 
total  OEs  as  the  dependent  measure.  (In  this  case, 
Table  9  was  not  used  for  screening  given  that  the 
stepwise  procedures  screens  for  the  contribution  of 
variables  to  the  analysis.)  The  results  shown  in  Table 
1  2  suggest  that  it  is  possible  to  explain  OE  incidence 
in  the  Atlanta  ARTCC  sectors  by  evaluating  fre¬ 
quency  congestion  and  the  influence  of  restricted 
airspace.  The  amount  of  variance  in  OE  incidence 
accounted  for  by  these  two  factors  was  3 1  %.  (With  three 
outliers  removed,  this  increased  to  45%,/  =  .000.) 

Discriminant  analysis  was  used  as  an  alternative 
approach  for  predicting  which  sectors  would  have 
errors.  Similar  to  the  MANOVA  discussed  earlier,  it 
determined  how  well  the  relevant  and  permissible 
variables  in  Section  4.1  distinguished  between  no¬ 
error,  low-error,  and  high-error  sectors.  Discriminant 
analysis  has  the  advantage  of  providing  success  rates 
for  predicting  group  membership. 

All  of  the  applicable  and  permissible  ATC  com¬ 
plexity  variables  were  made  available  for  the  proce¬ 
dure.  Applying  a  stepwise  approach  yielded  two 
discriminant  functions,  with  a  combin  cd  x'(4)  = 
18.33,/  =  .001.  The  first  function  provided  most  of 
the  discriminating  power;  the  second  was  not  signifi¬ 
cant  when  the  first  was  removed.  The  first  function 
accounted  for  33To  of  the  variability  in  the  grouping 
data.  Frequency  congestion  and  the  effect  of  restricted 
airspace  were  the  only  two  statistically  significant 
factors  entered  into  the  equation  (/  <  .01). 
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when  using  the  resulting  equation  to  predict  group 
membership,  there  was  an  overall  58%  success  rate. 
Notably,  the  individual  success  rates  for  predicting 
whether  a  sector  would  have  low  or  high  OE  rates  were 
68%  and  63%,  respectively.  The  formula  correctly 
predicted  that  a  sector  would  have  zero  errors  for  40% 
of  the  cases.  The  rates  of  correctly  identifying  sectors 
from  each  group  by  chance  would  be  33%,  48%,  and 
1 8%,  respectively,  for  the  no,  low,  and  high-error  groups. 

4.5.4  Situation  Awareness  and  Operational 
Errors 

For  each  of  the  85  OEs  in  the  database  from  Atlanta 
ARTCC,  a  QA  Specialist  had  made  a  rating  of  the 
presence  or  absence  of  SA.  Analyses  of  the  OE  and 
sector  data  were  made  to  investigate  whether  control¬ 
ler  SA  was  related  to  any  characteristics  of  errors  or  the 
sectors  in  which  they  occurred. 

As  in  Section  4.1.3,  Spearman  correlations  were 
computed  between  relevant,  OE-related  variables  from 
Section  4.2  to  screen  them  for  a  MAN  OVA  with  SA 
category  as  the  independent  variable.  The  only  signifi¬ 
cant  correlation  with  SA  category  was  for  horizontal 
separation,  r  =  .30, =  .005.  A  ^-test  for  horizontal 
separation  as  a  function  of  SA  category  showed  sig¬ 
nificantly  lower  separation  for  errors  in  the  no-SA 
group,  ^(54.69)  =  -3.06, =  .003  (A/=  3.39  versus  Af 
=  3.97).  A  chi  square  test  showed  no  significant 
difference  between  no-SA  and  SA  groups  as  a  function 
of  sector  type  (low,  high,  or  ultra  high). 

Further  analyses  considered  the  sector  characteris¬ 
tics  associated  with  each  error.  Therefore,  the  com¬ 
bined  set  of  sector  and  OE  data  was  used.  The  statistical 
test  that  addressed  sector  characteristics  as  a  function 
of  SA  had  to  take  into  account  that  more  than  one  OE 
occurred  in  many  sectors.  So  if  several  errors  were 
found  in  a  given  sector,  that  sector’s  characteristics 
were  weighted  more  heavily  in  the  analyses  than  with 
sectors  where  there  were  few  errors. 

Spearman  correlations  between  sector  characteris¬ 
tics  in  Section  4.1  and  SA  group  (conducted  for 
screening  purposes)  revealed  that  only  military  traffic 
and  military  operating  areas  had  low  correlations,  r  = 
-.19,/  =  .085  and  r  =  --19,/  =  .082,  respectively,  t- 
tests  were  conducted  and  significant  differences  were 
found  (at /  <  .1)  for  military  traffic,  ^(83)  =  1.78,/  = 


.079  (no  SA,  M  =  2.98  and  SA,  M  =  2.39)  and  for 
military  operating  areas,  ^(83)  =  1 .70,/  =  .093  (no  SA, 
M=  2.94  and  SA,  Af  =  2.13).  This  indicates  that  there 
were  few  outstanding  contrasts  in  sector  measures  as 
a  function  of  SA. 

However,  when  OE  frequency  with  and  without 
SA  was  plotted  against  overall  sector  errors,  a  trend 
was  evident,  as  shown  in  Figure  1 8.  (Sector  names  and 
OE  totals  are  shown  on  the  x-axis.)  In  this  analysis, 
sectors  with  one  error  were  omitted.  (Controllers  were 
always  aware  in  these  cases.)  It  appears  that,  as  the 
number  of  errors  in  sectors  increased,  there  were  more 
errors  of  which  the  controller  had  no  SA. 

The  correlation  of  overall  error  count  with  the 
number  of  no-SA  errors  was  r  =  .83,/  =  .000  and  with 
SA  errors  was  r-. 33,/=. 130. For  the  high  SA  sectors, 
the  correlation  of  overall  OE  count  with  non-SA 
errors  was  =  .86,  p  =  .006  and  with  SA  errors  was 
r  =  .36,  /  =  .386.  These  results  confirm  the  trend 
evident  in  Figure  18  that  error-prone  sectors  have 
many  errors  where  there  is  no  awareness  of  the  devel¬ 
oping  problem.  Or,  as  overall  OE  rate  increases,  no- 
SA  error  rate  rises  faster  than  SA  error  rate. 

Referring  to  Table  10,  there  were  a  number  of 
statistically  significant  and  near  significant  differ¬ 
ences  between  low  and  high  OE  sectors.  This  suggests 
that  some  sector  characteristics  may  negatively  affect 
SA,  which  in  turn  leads  to  higher  OE  rates. 

4.6  Summary  and  Discussion 

The  preceding  analysis  of  OE  data  will  be  summa¬ 
rized  in  this  section.  The  results  will  then  be  examined 
in  terms  of  questions  and  predictions  identified  in  the 
literature  review. 

4.6.1  Current  Findings 

There  was  an  average  of  two  errors  per  sector  at  the 
Atlanta  ARTCC  during  the  period  (June  1992  to  June 
1995),  with  one  sector  (Burne)  having  nine  errors.  The 
45  sectors  in  the  facility  varied  widely  in  volume,  with  a 
median  of  1 1 ,314  cu  mi.  Twenty-three  sectors  were  low 
(0  to  FL  230),  13  were  high  (FL240  to  FL340),  and  9 
were  ultra  high  (FL350  and  above).  Average  traffic 
density  (based  on  average  traffic  volume  in  each  sector 
over  a  three-day  period  in  the  1995  annual  facility 
review),  was  6.1  aircraft  per  sector. 
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A  factor  analysis  of  26  of  the  sector  characteristic 
variables  indicated  3  possible  underlying  dimensions: 
Traffic  Activity,  Size,  and  Military.  Three  further 
factors  were  identified  but  did  not  account  for  much 
of  the  original  information.  However,  the  annual 
review  measure  of  average  traffic  density  did  not 
appear  to  be  closely  related  to  other  estimates  of  traffic 
volume  (such  as  16CF  ratings  of  frequency  congestion 
and  traffic  volume),  suggesting  that  it  may  not  be 
performing  as  expected. 

A  review  of  the  OE  database  for  the  85  errors 
collected  between  1992  and  1995  shows  that  81%  of 
the  overall  causal  factors  were  attributed  to  problems 
with  the  radar  display,  29%  were  assigned  to  commu¬ 
nication  errors,  and  11%  to  coordination. 

OEs  at  the  Atlanta  ARTCC  during  the  reporting 
period  mostly  took  place  between  0800  and  2000  hr 
and  concentrations  were  found  between  15,000  and 
20,000  ft  and  between  FL300  and  FL350.  Given  that 
separation  standards  change  at  FL290,  the  flight  levels 
of  OEs  tended  to  be  distributed  into  two  groups,  accord¬ 
ing  to  the  zone  in  which  they  occurred.  A  disproportion¬ 
ately  high  number  oferrors  occurred  in  high  sectors,  with 
fewer  than  expected  in  ultra  high  sectors. 

Mean  traffic  volume  at  the  time  of  an  OE  was 
significantly  higher  than  as  reported  by  the  1995 
annual  review  of  average  sector  density.  The  average 
density  of  8.0  aircraft  when  an  error  occurred  was 
nearly  one  standard  deviation  above  the  base  rate  level 
of  6. 5  aircraft.  Judgments  of  complexity  at  the  time  of 
an  OE  averaged  3.4  (on  a  scale  of  5),  and  most  errors 
were  rated  as  occurring  in  moderately  complex  condi¬ 
tions.  A  workload  measure  derived  from  the  traffic 
density  and  complexity  data  indicated  that  some  er¬ 
rors  occurred  at  above  average  workload  levels,  as 
compared  to  the  whole  set  of  OEs.  Normative  or 
baseline  workload  data  for  Atlanta  ARTCC  sectors 
were  not  available  for  comparison. 

The  minimum  horizontal  and  vertical  separation 
distance  at  the  time  of  the  error  were  further  analyzed 
by  considering  RMS  distances.  Although  there  are 
different  vertical  separation  requirements  above  and 
below  FL290,  the  1000  ft  of  additional  separation 
only  creates  another  50  ft  of  straight  line  distance. 
Most  errors  were  found  to  have  at  least  20,000  ft  (or 
about  3.3  mi)  of  RMS  distance  remaining. 


Conflict  severity  was  calculated  for  the  OEs  and  it 
was  found  that  84%  were  moderate  and  16%  were 
severe.  There  was  a  moderate  correlation  between 
sector  type  (or  level)  and  severity,  suggesting  that 
more  severe  errors  occurred  in  sectors  below  FL230. 

Most  OEs  happened  with  one  or  two  controllers 
working.  Without  normative  information,  it  is  diffi¬ 
cult  to  know  the  proportion  of  time  that  one,  two,  or 
three  controllers  typically  work  a  position.  In  73%  of 
the  cases,  the  radar  controller  was  not  aware  of  the 
developing  OE. 

Sectors  in  the  Atlanta  ARTCC  were  divided  in  no¬ 
error,  low-error,  and  high-error  groups.  Bivariate  cor¬ 
relations  were  calculated  between  most  of  the  29 
original  sector  characteristic  variables  and  the  OE 
frequency  measures  (number  of  OEs  and  OE  group). 
Fifteen  variables  had  correlations  at  p  <  A  with  OE 
group.  Given  a  significant  MANOVA,  further  analy¬ 
ses  determined  that  there  were  statistically  significant 
differences  between  the  no-error  and  high-error  groups 
on  four  sector  variables  including  frequency  of  prob¬ 
lematic  weather,  radio  frequency  congestion,  total 
complexity,  and  average  complexity.  There  were  mar¬ 
ginally  significant  differences  (indicating  possible 
trends  in  the  data)  for  amount  of  climbing/descend¬ 
ing  traffic,  degree  of  aircraft  mix,  number  of  required 
procedures,  and  average  traffic  volume.  In  general, 
these  results  suggested  that  sectors  with  high  error 
counts  were  more  complex  than  those  with  no  errors. 

All  measures  in  the  AN OVA  increased  as  a  function 
of  error  count,  except  for  aircraft  mix  which  de¬ 
creased.  A  more  homogenous  (low  mix)  traffic  pattern 
composed  of  high-speed  jets  could  pose  more  chal¬ 
lenges  for  maintaining  separation. 

While  sector  size  (as  measured  by  OpenCrcate)  did 
not  emerge  as  a  significant  variable  in  the  ANOVAs, 
inspection  of  the  size  differences  between  OE  fre¬ 
quency  groups  showed  a  relationship,  and  this  was 
supported  by  a  statistical  test.  It  is  clear  that  high-error 
sectors  were  only  about  32%  of  the  size  of  no-  and 
low-error  sectors.  More  errors  occurred  in  lower, 
smaller  sectors.  It  is  not  surprising  that  smaller  sector 
size  would  induce  more  complexity. 

ANOVAs  were  used  to  consider  the  differences  in 
OE  characteristics  between  low  and  high  OE  sectors. 
First,  correlations  were  found  between  OE  group  (low 
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or  high)  and  number  of  aircraft  in  the  sector  (at  the 
time  of  the  error),  complexity  (as  rated  at  the  time  of 
the  error),  workload  index,  and  whether  the  position 
was  combined  at  the  time  of  the  error.  It  was  found 
that  the  high-error  sectors  had  more  aircraft,  were 
more  complex,  and  had  higher  workload.  High-error 
sectors  were  also  less  likely  to  have  positions  com¬ 
bined  than  the  low  sectors. 

Several  approaches  were  used  to  attempt  to  predict 
OE  occurrence  from  sector  and  traffic  variables.  Us¬ 
ing  multiple  regression,  it  was  possible  to  account  for 
3 1  %  of  the  variance  in  the  total  sector  errors  using  the 
sector  characteristics  of  radio  frequency  congestion 
and  effects  of  restricted  areas.  A  discriminant  analysis 
using  frequency  congestion  and  the  effect  of  restricted 
areas  resulted  in  a  formula  that  was  able  to  classify  sectors 
into  error  frequency  groups  with  58%  average  accuracy. 

OEs  were  separated  into  those  where  the  primary 
controller  had  awareness  of  the  developing  problem, 
as  opposed  to  those  where  no  awareness  was  present. 
The  only  OE  characteristic  that  was  significantly 
different  between  SA  error  groups  was  horizontal 
separation  which  was  greater  when  SA  was  present. 

Using  sector  variables,  significant  differences  were 
found  between  SA  and  no-SA  OEs  for  amount  of 
military  traffic  and  the  effect  of  military  operating 
areas,  with  no-SA  OEs  tending  to  occur  with  more  of 
each.  As  the  frequency  of  errors  within  individual 
sectors  increased,  there  was  evidence  that  SA  dimin¬ 
ished.  In  high-error  sectors,  there  was  a  general  reduc¬ 
tion  in  awareness  of  errors.  Thus,  those  sector 
characteristics  listed  previously  that  discriminated 
between  low-  and  high-error  sectors  also  probably 
have  a  relationship  to  SA. 

1,6.2  Relationships  to  Previous  Research 

The  literature  review  identified  many  factors  pos¬ 
sibly  related  to  OE  or  conflict  occurrence.  Table  4  has 
been  reproduced  in  Table  13  to  list  the  findings  from 
this  study  that  address  the  issues  raised  in  the  litera¬ 
ture  review.  (NS  indicates  no  statistically  significant 
results.) 

Although  it  was  possible  to  summarize  the  primary 
and  secondary  causes  attributed  to  operational  errors 
in  the  1992  to  1995  data,  the  rating  system  had 
changed  so  that  direct  comparisons  to  Kinney,  et  al. 


(1977),  Stager  and  Hameluck  (1990),  and  Stager,  et 
al.  (1989)  were  not  feasible.  However,  Schroeder 
(1982)  noted  that  coordination  was  cited  as  a  contrib¬ 
uting  factor  in  27  to  54%  of  the  OE  reports  between 
1969  and  1980.  Redding  (1992)  apparently  used  the 
more  recent  classification  system  and  found  that 
misidentification  or  misuse  of  radar  data  was  cited  in 
38%  of  the  1989  error  reports.  Schroeder  and  Nye 
(1993)  employed  the  same  categories  for  1985  to 
1988  data.  Table  14  shows  a  comparison  of  the 
Schroeder  and  Nye  (1992)  results  with  the  1992  to 
1995  data. 

A  larger  proportion  of  errors  in  the  current  study 
were  attributed  to  problems  with  the  radar  display 
than  in  previous  reviews.  Communication  problems 
remained  about  the  same  as  in  the  Schroeder  and  Nye 
(1993)  report,  but  fewer  errors  were  assigned  to  prob¬ 
lems  with  coordination  or  data  posting. 

Many  factors  were  suggested  in  the  literature  re¬ 
view  as  being  possibly  related  to  OEs  (Arad,  1964; 
Schmidt,  1976;  Couluris  and  Schmidt,  1973;  Stein, 
1985;  Buckley,  et  al.  1983;  Fowler,  1980;  Grossberg, 
1989;  Mogford,  et  al.,  1993;  Empson,  1987;  Langan- 
Fox  and  Empson,  1985).  These  are  listed  in  Table  13. 
The  variables  in  the  current  study  that  were  found  to 
be  correlated  with  OE  rate  or  that  distinguished 
between  high  and  low  OE  sectors  are  linked,  where 
possible,  to  the  specific  issues  raised  in  the  literature 
review.  In  many  cases,  there  is  some  correspondence, 
indicating  that  some  of  the  findings  from  the  current 
research  directly  support  (or  in  some  cases  fail  to 
corroborate)  previous  work  in  this  area. 

Several  studies  noted  that  most  OEs  occur  during 
times  of  moderate  workload  or  traffic  volume  (Kinney, 
et  al.,  1977;  Schroeder,  1982;  Stager  and  Hameluck 
(1989,  1990);  Redding  (1992);  Schroeder  and  Nye, 
1993.)  While  no  direct  ratings  of  workload  were 
available  for  the  current  set  of  errors,  there  were 
indications  that  errors  occurred  under  higher  than 
normal  traffic  densities.  The  average  traffic  load  at  the 
time  of  an  OE  was  eight  aircraft,  the  same  number 
found  by  Redding  (1992).  Rodgers  and  Nye  (1993) 
found  that  the  average  traffic  load  for  1988  to  1991 
OEs  was  8.8  aircraft.  However,  indirect  corrobora¬ 
tion  of  the  findings  regarding  moderate  workload 
were  found  in  the  complexity  ratings  given  each  OE. 
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The  average  complexity  was  3.4  (out  of  5),  and  most 
errors  were  in  the  moderate  range.  When  a  derived 
workload  measure  was  calculated,  it  appeared  that 
OEs  occurring  in  high-error  sectors  had  marginally 
higher  workload. 

Although  several  authors  suggested  that  coordina¬ 
tion  could  be  a  factor  in  OE  incidence  (Arad,  et  al., 
1964;  Couluris  and  Schmidt,  1973;  Stein,  1985; 
Fowler,  1980;  Mogford,  et  al,,  1993;  Rodgers  and 
Manning,  1995)>  there  was  not  much  evidence  in  the 
current  data  set  to  support  this  hypothesis.  The  only 
measure  of  the  amount  of  coordination  required  in  a 
sector  was  one  item  on  the  1 6CF,  and  this  variable  was 
not  correlated  with  OE  frequency  nor  did  it  discrimi¬ 
nate  between  groups  of  sectors  with  different  OE 
severities.  As  noted  earlier,  coordination  was  men¬ 
tioned  as  a  factor  in  OE  occurrence  in  11%  of  the 
reports  during  the  period.  However,  as  a  whole,  the 
data  do  not  add  much  to  our  understanding  about  the 
effects  of  sector  boundary  placement  and  coordina¬ 
tion  on  OE  occurrence. 

In  the  literature  review,  some  reports  indicated  that 
conflicts  usually  occurred  between  aircraft  in  level 
flight  (Siddiqee,  1973;  Kinney,  et  al.  1977).  Rodgers 
and  Nye  (1993)  found  that  most  errors  involved  one 
aircraft  in  level  flight,  and  one  that  was  climbing  or 
descending,  though  most  moderately  severe  errors 
involved  aircraft  in  level  flight.  The  OE  data  reviewed 
in  this  report  did  not  contain  information  relevant  to 
this  topic.  However,  there  was  a  low,  but  significant 
correlation  between  number  of  OEs  and  amount  of 
climbing  and  descending  traffic.  This  suggests  that 
transitioning  traffic  had  some  role  in  error  generation. 

Rodgers  and  Nye  (1993)  also  found  that  30%  of 
the  1053  errors  they  analyzed  were  rated  as  moderate 
in  severity  while  70%  were  minor  errors.  In  compari¬ 
son,  the  current  data  show  16%  moderate  and  84% 
minor  errors,  with  a  much  smaller  set  of  85  errors. 

Finally,  there  is  unfortunately  not  much  relation¬ 
ship  between  the  approach  used  in  this  research  and 
the  recent  work  by  Rodgers  and  Manning  (1995). 
Aircraft  density  was  generally  higher  at  the  time  of  an 
OE,  but  it  was  not  possible  to  address  the  other  variables 
used  in  this  previous  study,  given  the  data  at  hand. 


5.  CONCLUSIONS 

Several  useful  and  important  findings  emerged  as  a 
result  of  analyzing  the  1 992  to  1 995  operational  error 
(OE)  reports  and  investigating  relationships  between 
sector  and  traffic  factors  and  OEs.  These  will  be 
discussed  in  this  section  and  recommendations  for 
further  study  will  be  enumerated. 

Using  a  combination  of  OpenCreate  and  other  data, 
it  was  possible  to  generate  useful  statistics  regarding 
Atlanta  Air  Route  Traffic  Control  Center  (ARTCC) 
sector  features.  Of  note  was  the  wide  range  in  sector  size. 
These  descriptive  data  may  be  useful  for  comparing  en 
route  facilities  during  future  research  efforts. 

Without  normative  data  on  the  daily  traffic  flows 
and  typical  altitudes,  it  is  difficult  to  determine  whether 
OE  frequency  departed  from  expected  proportions,  as 
determined  by  normal  traffic  density  patterns.  It  may 
also  be  that  ultra-high  sectors  are  often  combined 
with  high  sectors.  However,  it  appears  that  more  OEs 
occurred  in  high  sectors,  and  fewer  in  ultra-high 
sectors,  than  would  be  anticipated,  based  on  sector 
counts.  Future  research  should  gather  facility  baseline 
data  to  support  such  comparisons. 

It  is  evident  that  errors  tended  to  occur  at  above 
1995  average  traffic  density.  However,  it  is  again 
difficult  to  know  how  much  busier  these  sectors  were 
relative  to  normal  ranges  of  traffic  density  at  the  time 
of  OEs.  As  found  by  Kinney,  et  al.  (1977)  and  others, 
the  traffic  noted  in  the  OE  reports  may  fall  within  the 
moderate  range.  Furtherdata  from  the  Atlanta  ARTCC 
would  help  determine  whether  OE  frequency  de¬ 
parted  from  expected  proportions,  as  determined  by 
normal  traffic  density  patterns. 

OE  complexity  rating  (from  the  OE  report)  had  a 
high  correlation  with  aircraft  density  in  the  sector  at 
the  time  of  the  OE.  It  had  much  lower  correlations 
with  other  general  sector  volume  and  complexity 
ratings.  This  suggests  that  more  immediate  factors, 
such  as  traffic  density  during  the  OE  period,  arc 
important  in  this  rating,  as  compared  to  overall  sector 
characteristics. 

Of  interest  was  the  moderate  correlation  of  error 
severity  and  sector  type  (L,  H,  or  UH),  indicating  that 
sectors  under  FL230  tend  to  have  more  severe  errors. 
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This  result  is  nearly  identical  to  a  finding  by  Rodgers 
and  Nye  (1993)  that  more  severe  errors  occurred 
below  FL290. 

A  factor  analysis  using  OpenCreate  and  sixteen 
complexity  factor  (16CF)  variables  indicated  three 
primary  underlying  dimensions:  Traffic  Activity,  Size, 
and  Military.  The  other  three  factors  in  the  analysis 
showed  no  strong  patterns,  with  one  factor  lacking 
any  distinct  loadings.  It  is  reassuring  to  observe  that 
the  average  complexity  facility  review  measure  was 
associated  with  other  Traffic  Activity  variables  and 
that  the  sector  size  variables,  as  measured  by 
OpenCreate  and  16CF,  were  related.  However,  the 
facility  review  traffic  density  variable  was  not  loaded 
on  the  same  factor  as  other  traffic  density  indicators. 
This  suggests  that  it  may  lack  concurrent  validity. 
Fortunately,  all  other  volume-related  measures  were 
loaded  on  the  Traffic  Activity  factor.  It  would  be 
helpful  to  evaluate  traffic  density  in  the  Atlanta 
ARTCC  sectors  in  additional  ways  to  better  define  the 
nature  of  the  average  density  measure. 

Traffic  mix  was  associated  with  the  Size  factor, 
rather  than  with  Traffic  Activity.  The  presence  of  a 
negative  weighting  of  aircraft  mix  on  this  factor  sug¬ 
gests  that  a  preponderance  of  larger  jet  aircraft  may  be 
associated  with  larger  sectors.  The  collection  of  data 
on  aircraft  type  for  each  OE  would  be  useful  in  this 
analysis.  Adequacy  of  radio/radar  coverage  fell  in  with 
the  Military  factor.  Areas  that  have  poor  radio  and 
radar  coverage  have  some  similarity  to  military  air¬ 
space  in  that  they  are  less  accessible  to  FAA  air  traffic 
control.  Finally,  the  sector  shape  measure,  percent  of 
cube,  seemed  to  have  no  relationship  to  other  measures. 
This  could  indicate  that  it  taps  some  independent  factor. 
However,  by  accounting  for  only  7%  of  the  variance  in 
the  data  set,  it  may  be  of  limited  importance. 

The  potential  for  a  reduction  in  sector  measures  to 
a  set  of  three  or  four  factors  holds  promise  in  that  it 
might  simplify  the  process  of  evaluating  sector  char¬ 
acteristics.  The  practice  would  be  to  use  one  or  more 
of  the  variables  that  were  loaded  on  each  distinct 
factor.  However,  given  that  these  variables  do  not 


fully  represent  each  factor,  some  information  is  lost. 
Further  research  would  be  needed  to  determine  the 
utility  of  this  approach. 

A  theoretical  issue  worth  noting  is  that,  given  the 
wide  range  of  ATC  complexity  measures  available, 
there  appear  to  be  only  three  general  characteristics  of 
sectors  related  to  traffic  activity,  size,  and  military 
operations.  This  makes  intuitive  and  practical  sense, 
given  controllers'  descriptions  of  sector  workload 
issues.  The  level  of  difficulty  of  an  air  traffic  situation 
is  often  described  as  an  interaction  between  sector  size 
and  the  amount  and  behavior  of  traffic.  This  relation¬ 
ship  was  also  found  in  the  Buckley  et  al.  (1983)  work 
where  sector  geometry  and  traffic  density  interacted 
to  affect  controller  performance. 

Stein's  (1985)  research  showed  that  variables  re¬ 
lated  to  the  traffic  activity  and  size  factors  accounted 
for  a  large  proportion  of  controller  workload.  In  the 
current  study.  Military  was  discovered  to  be  an  addi¬ 
tional  independent  factor.  This  seems  reasonable  in 
that  military  aircraft  and  their  associated  airspace 
reservations  are  controlled  by  outside  agents  and  must 
be  accommodated  by  the  controller  in  different  ways 
than  are  commercial  aircraft,  weather,  and  other  centers. 

An  application  of  the  factor  analysis  findings  could 
be  used  as  a  general  guideline  for  traffic  management 
systems.  Aircraft  activity,  sector  size,  and  military 
operations  should  be  included  in  any  formula  that 
seeks  to  account  for  or  predict  ATC  complexity. 

When  comparing  sectors  with  no  errors,  less  than 
three  errors,  and  four  or  more  errors,  there  was  consis¬ 
tent  evidence  supporting  the  role  of  ATC  complexity. 
First,  1 5  of  the  sector  and  traffic-related  variables  were 
correlated  with  OE  frequency  or  OE  frequency  group. 
Also,  it  was  found  that  there  were  differences  between 
OE  frequency  groups  on  a  number  of  measures.  Five 
variables  (including  sector  size)  showed  statistically 
significant  differences  between  groups,  and  two  of 
these  were  general  complexity  measures.  Three  other 
variables  demonstrated  trends  in  the  same  direction 
(increasing  errors  with  higher  complexity).  This  demon¬ 
strates  a  definite  role  for  ATC  complexity  in  OE  analysis. 


Although  many  of  the  correlations  found  were  in  the  .30  range,  and  therefore  not  particularly  strong,  many  were  statistically  significant. 
Correlations  in  this  range  are  acceptable  for  this  type  of  exploratory  research. 
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It  is  also  interesting  to  note  that  three  of  the 
variables  that  were  statistically  significant,  and  all  four 
of  the  marginal  variables  were  part  of  the  T  raffic  Activity 
factor.  Thus,  this  factor  alone  may  account  for  many  of 
the  effects  that  contribute  to  OE  occurrence. 

One  of  the  potential  benefits  of  collecting  data  on 
factors  that  might  be  related  to  OE  occurrence  is  that 
the  ability  to  predict  these  events  could  improve. 
This,  in  turn,  would  pave  the  way  for  the  development 
of  practical  tools  for  conflict  management  that  might 
be  used  in  en  route  ATC  operations.  Several  tech¬ 
niques  were  applied  to  explore  this  possibility,  all 
using  the  same  underlying  statistical  approach. 

Multiple  regression,  which  was  used  to  build  the 
optimal  mathematical  combination  of  variables  that 
will  predict  the  number  of  OEs  in  a  given  sector, 
accounted  for  about  3 1  percent  of  the  available  infor¬ 
mation.  The  size  of  this  correlation  is  respectable  on 
theoretical  grounds  in  that  it  demonstrates  that  there 
is  a  firm  relationship  between  sector  characteristics 
and  OE  rate.  However,  it  is  not  of  much  practical 
importance  in  that  the  equation,  in  its  present  form, 
will  not  permit  accurate  conflict  prediction. 

The  fact  that  only  two  variables  (frequency  conges¬ 
tion  and  restricted  areas)  entered  the  regression  analy¬ 
sis  suggests  that  there  may  be  a  significant  amount  of 
redundant  information  in  the  data  set.  (This  can  been 
seen  in  the  correlations  in  Appendix  A.)  It  is  interest¬ 
ing  to  note  that  one  of  these  variables,  frequency 
congestion,  also  appeared  in  the  multiple  correlation 
with  overall  ATC  complexity  derived  by  Mogford,  et 
al.  (1993).  The  two  variables  were  members  of  the 
Traffic  Activity  and  Military  factors,  respectively. 

Although  the  regression  analysis  ultimately  em¬ 
ployed  only  two  variables,  this  does  not  necessarily 
imply  that  these  are  the  only  meaningful  factors  for 
further  study.  The  other  sector  characteristics  also 
have  useful  information,  but  their  inter-correlations 
suggest  that  there  may  be  only  a  few  underlying 
themes,  as  demonstrated  in  the  factor  analysis.  If 
description  of  sectors  is  the  goal,  these  variables  should 
be  retained,  for  they  provide  a  richness  of  detail.  For 
predictive  purposes,  however,  it  may  be  adequate  to 
employ  a  subset  of  the  original  measures. 


Discriminant  analysis  was  applied  in  an  attempt  to 
use  the  available  variables  to  predict  whether  a  sector 
belonged  to  the  no,  low,  or  high-error  groups.  Using 
the  original  measures,  frequency  congestion  and  re¬ 
stricted  areas  again  emerged  as  the  only  two  emergent 
factors,  and  it  was  possible  to  achieve  an  average 
classification  accuracy  rate  of  58%.  This  resulted  in  a 
success  rate  of40%,  68%,  and  63%  for  the  no-,  low-,  and 
high-error  groups,  respectively.  The  chance  rates  of 
correctly  identifying  sectors  from  each  group  would 
be  33%,  48%,  and  18%,  respectively,  for  the  no-, 
low-,  and  high-error  groups.  Thus,  the  discriminant 
analysis  adds  predictive  power,  especially  for  identify¬ 
ing  potential  high-error  sectors. 

Practically  speaking,  the  approach  reflected  in  cor¬ 
relation  or  multiple  regression  might  be  the  most 
useful  in  ATC  operations.  The  results  of  such  analyses 
would  indicate  that  OEs  would  be  more  likely  to 
occur  with  the  increase  or  decrease  in  certain  dynamic 
sector  or  traffic-related  factors.  Such  information 
might  assist  flow  controllers  and  area  supervisors  in 
taking  steps  to  avoid  problems  before  they  develop. 
Another  application  could  be  to  assist  with  defining 
such  free  flight  concepts  as  dynamic  density  and 
flexible  resectorization. 

There  were  also  static  measures,  such  as  sector  size, 
that  contributed  toward  high  error  frequency.  How¬ 
ever,  from  experience,  facility  personnel  already  are 
familiar  with  which  sectors  are  error-prone.  Being 
able  to  group  sectors  by  error  frequency,  as  predicted 
by  a  set  of  sector  and  traffic  measures,  might  be  more 
useful  for  airspace  reconfiguration  projects.  Another 
application  could  be  to  assist  with  dynamic 
resectorization,  a  concept  proposed  for  free  flight. 

It  would  be  desirable  to  determine  when  high-risk 
situations  are  developing  and  predict  potentially  prob¬ 
lematic  sectors.  Using  information  on  sector  charac¬ 
teristics,  the  methods  used  here  show  some  promise, 
but  are  not  as  yet  sufficiently  powerful.  The  16CF 
questionnaire  needs  further  validation  to  ensure  that 
it  is  performing  as  intended.  Some  evidence  of  con¬ 
current  validity  for  some  of  the  items  was  found  in  the 
factor  analysis.  It  would  be  interesting  to  create  some 
additional  complexity  questionnaire  items  for  evaluation 
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and  add  variables  to  OpenCreate.  Based  on  the  litera¬ 
ture  review,  some  candidate  measures  might  be:  a 
count  of  airway  crossings  of  sector  boundaries  (to 
evaluate  coordination  requirements),  aircraft  speeds 
at  the  time  of  the  OE,  route  complexity,  a  count  of 
event-  versus  controller-driven  activity  (Langan-Fox 
and  Empson,  1985),  and  flight  stage  (climbing,  descend¬ 
ing,  or  level)  at  the  time  of  the  OE.  The  characteristics 
inherent  in  the  dimensions  of  the  factor  analysis  could 
also  be  melded  into  more  global  questions. 

Collection  of  additional  ratings  of  ATC  complex¬ 
ity  from  Atlanta  Center  might  help  stabilize  the  1 6CF 
data.  (Only  one  controller  from  each  area  was  used  in 
this  study.)  Given  that  previous  research  found  that 
interrater  reliability  between  controllers  from  the  same 
areas  tended  to  be  only  moderate,  further  steps  should 
be  taken  to  ensure  clear  definition  of  each  factor, 
perhaps  using  graphics  combined  with  written  de¬ 
scriptions.  A  computerized  version  of  the  16CF  ques¬ 
tions  may  be  worth  exploring. 

To  summarize,  this  research  has  shown  that  high- 
OE  sectors  are  characterized  by  problematic  weather, 
radio  frequency  congestion,  high  total  16CF  com¬ 
plexity,  high  annual  review  average  complexity,  and 
small  size.  There  is  also  evidence  that  these  sectors 
tend  to  have  more  climbing/descending  traffic,  a 
uniform  aircraft  mix,  frequent  required  procedures, 
and  higher  traffic  volume.  The  general  dimensions 
that  describe  sector  and  traffic  characteristics  are 
traffic  activity,  size,  and  military.  Finally,  there  is 
limited  evidence  that  OE  probability  can  be  predicted 
using  a  subset  of  these  variables. 

The  separate  analysis  of  controller  situational  aware¬ 
ness  (SA)  during  the  development  of  OEs  is  important 
in  that,  in  these  data,  73%  of  the  controllers  were  not 
aware  of  the  developing  error.  Presumably,  awareness 
would  have  prevented  many  of  these  errors  from 
occurring,  as  suggested  by  Redding  (1992).  The  only 
sector  or  traffic  characteristic  that  was  clearly  differ¬ 
ent  between  not  aware  and  aware  OE  groups  was 
horizontal  separation,  as  found  by  Rodgers  and  Nye 
(1993).  However,  this  result  may  have  been  due  to  the 
fact  that,  without  awareness,  the  error  was  more  fully 
advanced  before  controller  intervention  occurred. 


It  was  found  that  high-error  sectors  tended  to  have 
more  no-SA  errors.  It  may  be  that  the  presence  of 
awareness  of  a  developing  error  is  a  mediating  factor 
controlling  the  frequency  and  severity  of  errors  in  a 
given  sector.  If  sector  or  traffic  characteristics  tend  to 
somehow  interfere  with  general  controller  SA,  it  can 
be  expected  that  more  errors  will  occur,  and  they  will 
often  be  rated  as  no-SA  OEs.  Thus,  higher  ATC 
complexity  may  result  in  the  kind  of  high  cognitive 
loading  that  contributes  to  a  reduction  in  SA  and  leads 
to  an  elevated  probability  of  error. 

Although  direct  comparisons  between  the  current 
findings  and  previous  research  in  this  area  were  not 
always  possible,  there  were  many  links.  It  was  also 
notable  that  there  was  a  24%  rise  in  the  attribution  of 
OE  causation  to  problems  with  the  radar  display. 
Coordination  and  data  posting  were  cited  less  fre¬ 
quently,  and  communication  remained  at  about  30%. 
This  represents  a  significant  increase  in  problems  with 
misreading  or  misusing  visually-displayed  data  with 
the  concomitant  negative  impact  on  SA.  A  more 
detailed  analysis  of  this  finding  is  indicated.  Changes 
to  information  display  methods  may  be  required  to 
eliminate  some  of  the  causes  of  these  errors. 

This  project  has  resulted  in  a  review  of  the  sector- 
and  traffic- related  factors  associated  with  OEs.  Armed 
with  OE  data  from  the  Atlanta  ARTCC  from  a  variety 
of  sources  (including  SATORI),  it  has  been  possible 
to  explore  relationships  between  a  range  of  ATC 
complexity  measures  and  errors.  Although  this  study 
must  be  considered  as  exploratory,  it  has  succeeding 
in  demonstrating  a  role  for  sector  complexity,  as 
measured  by  an  array  of  variables,  in  OE  incidence.  It 
has  also  confirmed  the  Rodgers  and  Nye  (1993)  asser¬ 
tion  that  SA  may  be  an  important  mediating  factor  in 
the  creation  and  eventual  severity  of  errors.  With  a 
larger  data  set  and  refined  measurement  techniques,  it 
may  be  possible  to  generate  reliable,  statistically  sound 
rules  for  the  prediction  of  OEs,  and  useful  guidelines 
for  sector  reconfiguration  and  design.  This  could 
provide  direct  benefits  for  the  development  of  proce¬ 
dures  and  automation  tools  for  reducing  errors  and 
enhancing  safety  in  the  NAS. 
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FIGURES 


MEDIATING  FACTORS 


FIGURE  1.  FACTORS  AFFECTING  CONTROLLER  WORKLOAD 
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FIGURE  2.  DISTRIBUTION  OF  SECTORS  BY  NUMBER  OF  OEs 
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3.  DISTRffiUTION  OF  SECTORS  BY  CUBIC  VOLUME  OF  AIRSPACE 


FIGURE  4.  DISTRIBUTION  OF  SECTORS  BY  TYPE 
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FIGURE  6.  HISTOGRAM  OF  OE  COUNT  BY  TIME  OF  DAY 
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FIGURE  7.  DISTRIBUTION  OF  OEs  BY  FLIGHT  LEVEL 


nCURE  8.  DISTRIBUTION  OF  OEs  BY  SECTOR  TYPE 
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FIGURE  10.  DISTRroUTION  OF  OEs  BY  TRAFFIC  COUNT 


29 


Number  of  OEs  \  Number  of  OEs 


FIGURE  11.  DISTRBUTION  OF  OEs  BY  COMPLEXITY  RATING 


FIGURE  12.  DISTRIBUTION  OF  OEs  BY  WORKLOAD  INDEX 
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FIGURE  13.  DISTRIBUTION  OF  OEs  BY  VERTICAL  SEPARATION 


FIGURE  14.  DISTRIBUTION  OF  OEs  BY  HORIZONTAL  SEPARATION 
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FIGURE  15.  MINIMUM  RMS  SEPARATION  FOR  OEs 
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FIGURE  16.  NUMBER  OF  CONTROLLERS  WORKING 
AT  THE  TIME  OF  THE  ERROR 
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FIGURE  17.  SECTOR  VOLUME  FOR  OE  ERROR  GROUPS 
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FIGURE  18.  OE  FREQUENCY  FOR  SA  AND  NO-SA  ERRORS  AS  A  FUNCTION  OF  TOTAL  OEs 
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TABLE  1.  NUMBER  OF  OPERATIONAL  ERRORS  AS  A  FUNCTION  OF  TRAFFIC 
VOLUME  AND  WORKLOAD  COMPLEXITY  IN  1974-76  SEIS  DATA 


Workload  Complexity 

Traffic  Volume 

Moderate 

Total 

Light 

127 

68 

9 

204 

Moderate 

22 

160 

70 

252 

Heavy 

0 

14 

73 

87 

Total 

149 

242 

152 

543 

TABLE  2.  MEASURES  USED  IN  THE  RODGERS  AND  MANNING  (1995)  STUDY 


Measure 

Type 

Variable 

Sector  Activity 

Handoff  Activity 

Number  of  Aircraft  Transiting 
Sector 

Average  Sector  Transit  Time 

Number  of  Handoffs 

Accepted 

Latency  to  Accept  Handoffs 

Host  computer  system 
(HCS)  Inputs 

Number  of  HCS  Inputs 

Number  of  Input  Errors 

Aircraft  Proximity 

Conflict  Alert 

Number  of  Conflict  Alerts 

Duration  of  Conflict  Alerts 

Proximity 

Average  Horizontal 

Separation 

Average  Vertical  Separation 

Number  of  Aircraft  Pairs 

Within  a  Criterion  Distance  of 
Each  Other 

Average  Time  Aircraft  Pairs 
Spent  Within  a  Criterion 
Distance  of  Each  Other 

Load 

Number  of  Aircraft  in  Sector 
by  Track  Type 

Aircraft  Dynamics 

Speed,  Altitude,  and 
Heading 

Average  Change 

Number  of  State  Changes 

Number  of  Aircraft  Changing 
State 

Number  of  Change  Over  a 
Criterion  Level 

TABLES.  MANOVA RESULTS 


Variable 

Result 

Average  Sector  Transit  Time 

Increase 

Average  Latency  to  Accept  Handoff 

Increase 

Number  of  Handoffs  Accepted 

Decrease 

Average  Vertical  Separation 

Increase 

Average  Time  Aircraft  Were  Within  10  mi  &  1000  ft 

Increase 
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TABLE  4.  SUMMARY  OF  LITERATURE  REVIEW 


Source 

Factors  or  Issues 

Arad  (1 964) 

Conflicts  related  to  rules  of  separation,  average  traffic  speed, 
number  of  aircraft  under  control,  sector  size,  and  flow 
organization. 

Arad,  et  al.  (1 964) 

Routine  load  on  controller  affected  by  placement  of  sector 
boundaries  with  respect  to  traffic  flow. 

Siddiqee  (1973) 

Conflicts  in  en  route  airspace  occur  due  to  a  loss  of  horizontal 
separation  between  aircraft  at  the  same  altitudes. 

Schmidt  (1976) 

Conflicts  predicted  by  traffic  flow  rate,  separation  standards,  route 
geometry,  aircraft  speed,  aircraft  flow  rate,  angle  of  airway 
intersection,  number  of  flight  levels,  and  amount  of  transitioning 
traffic. 

Couluris  and  Schmidt  (1973) 

Cost  of  sectorization  is  additional  workload  (coordination)  imposed 
by  placement  of  sector  boundaries. 

Empson  (1987) 

Lanqan-Fox  and  Empson  (1985) 

Controller  workload  is  related  to  airspace  structure,  procedural 
demands,  traffic  type,  and  control  over  task  presentation  rate. 

Kinney,  etal.  (1977) 

OEs  occur  under  low  to  moderate  workload  and  moderate 
complexity.  In  en  route  centers,  95%  of  errors  attributed  to 
attention,  judgment,  or  communications.  Most  errors  occur  in  level 
flight. 

Schroeder  (1 982) 

Most  errors  occur  under  light  or  moderate  workload.  Other 
aspects  of  the  situation  [sector  factors?],  apart  from  traffic  volume, 
determine  workload.  Coordination  is  a  direct  or  contributing  factor 
in  many  errors. 

Stager  and  Hameiuck  (1 990) 
Stager,  et  al.  (1989) 

Definitions  of  direct  and  contributing  causes.  OEs  occur  under  low 
to  moderate  workload  conditions.  Causes  are  attention,  judgment, 
and  communication  problems. 

Redding  (1992) 

Failure  to  maintain  SA  causes  most  errors  under  moderate  traffic 
load.  Communication,  coordination,  and  misuse  of  radar  data 
account  for  most  errors. 

Schroeder  and  Nye  (1 993) 

OEs  occur  under  average  or  lower  traffic  complexity.  Problems 
with  radar  display,  communication,  coordination,  and  data  posting 
most  frequent  causes. 

Rodgers  and  Nye  (1993) 

Most  OEs  occur  with  one  aircraft  in  level  flight  and  another 
descending  or  ascending.  Most  moderate  errors  are  between 
aircraft  in  level  flight.  Horizontal,  not  vertical,  separation  varies 
with  severity.  Higher  horizontal  separation  for  SA  OEs. 

Fowler  (1980) 

Sector  complexity  effected  by  problems  with  coordination, 
procedures,  LOAs,  and  weather. 

Buckley,  et  al.  (1983) 

Sector  geometry  and  traffic  density  interact  to  affect  controller 
performance. 

Stein  (1 985) 

Controller  workload  is  related  to  clustering  of  aircraft  in  a  small 
amount  of  airspace,  number  of  hand-offs  outbound/inbound,  and 
total  number  of  flights  handled. 
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TABLE  4.  SUMMARY  OF  LITERATURE  REVIEW  (CONTINUED) 


Source 

Factors  or  Issues 

Grossberg  (1989) 

Sector  complexity  factors  include  control  adjustments  to  merge 
and  space  aircraft,  climbing  and  descending  aircraft  flight  paths, 
mixture  of  aircraft  types,  frequent  coordination,  and  heavy  traffic. 

Mogford,  et  al.  (1993) 

ATC  complexity  factors  that  may  affect  controller  workload  (and 
OEs): 

1 .  Number  of  climbing  and  descending  aircraft. 

2.  Degree  of  aircraft  mix. 

3.  Number  of  intersecting  flight  paths. 

4.  Number  of  multiple  functions  controller  must  perform. 

5.  Number  of  required  procedures  controller  must  perform. 

6.  Number  of  military  flights. 

7.  Frequency  of  contacts  (coordination)  or  interface  with  other 
entities. 

8.  Extent  to  which  controller  is  affected  by  airline  hubbing. 

9.  Extent  to  which  controller  is  affected  by  weather. 

10.  Number  of  complex  aircraft  routings. 

1 1 .  Extent  to  which  controller  is  affected  by  restricted,  warning, 
and  military  operating  areas. 

12.  Size  of  sector  airspace. 

13.  Requirement  for  longitudinal  sequencing  and  spacing. 

14.  Adequacy  and  reliability  of  radio  and  radar  coverage. 

15.  Amount  of  radio  frequency  congestion. 

16.  Average  amount  of  traffic. 

Rodgers  and  Manning  (1995) 

OE  time  period  shows  increases  in  sector  transit  time,  handoff 
acceptance  latency,  vertical  separation,  and  aircraft  density  and 
decrease  in  number  of  handoffs  accepted. 
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TABLE  5.  FACTOR  ANALYSIS  RESULTS  FOR  SECTOR  CHARACTERISTICS 


Sector  Variable 

Factor  1 

Factor  2 

Factor  3 

Factor  4 

Factor  5 

Factor  6 

Variance  Accounted  For 

4% 

Number  of  VORTACs 

0.24 

0.21 

KSsIHlI 

0.01 

Number  of  shelves 

0.09 

-0.54 

0.44 

0.36 

Average  complexity 

KiBI 

KEfiHI 

M*im 

Average  density 

^ESIHI 

0.51 

keish 

IES3III 

Traffic  volume 

0.50 

I^B! 

0.06 

0.25 

-0.16 

Frequency  congestion 

0.48 

iEE3i 

-0.20 

0.12 

0.11 

Climbing/descending  traffic 

0.84 

-0.31 

■SOI 

-0.13 

Coordination 

0.48 

0.11 

mu 

0.05 

Multiple  functions 

|||7^^]||| 

KESH 

-0.11 

Hubbing 

0.18 

■sELI 

kbh 

-0.18 

Number  of  intersections 

0.34 

KE9i 

■iElsB 

HQI 

0.17 

Intersecting  flight  paths 

0.45 

i^g 

-0.20 

Complex  routings 

0.55 

KISfl 

— 

IEEEIII 

0.00 

-0.47 

0.00 

0.15 

Miles  of  victor  routes 

0.21 

Miles  of  other  routes 

-0.26 

0.86 

0.25 

-0.14 

Percent  of  cube 

0.15 

-0.16 

IKkH 

MEM 

0.05 

Military  traffic 

0.13 

-0.21 

-0.47 

HU 

0.23 

Restricted  areas 

0.10 

-0.47 

0.48 

-0.23 

0.21 

Aircraft  mix 

0.18 

0.07 

0.44 

-0.24 

Required  procedures 

0.19 

0.17 

-0.08 

-0.21 

Radar/radio  coverage 

0.36 

-0.59 

-0.10 

0.47 

Cubic  volume 

— 

0.61 

msm 

0.03 

lEI^I 

Sector  size 

0.61 

IBEEB 

keu 

0.47 

Sequencing  and  spacing 

0.57 

0.39 

-0.32 

0.06 

-0.11 

-0.09 

Weather 

0.66 

0.29 

0.11 

-0.08 

0.18 

0.10 

TABLE  6.  PRIMARY  CAUSAL  FACTORS 


Primary  Causal  Factor 

Number 

Percent 

Radar  display:  Inappropriate  use  of  displayed  data:  Other 

36 

42 

Communication  error 

15 

18 

Radar  display:  Misidentification:  Climbed  aircraft  with  similar  call  sign 

14 

16 

Radar  display:  Misidentification:  Failure  to  maintain  lateral  separation 

4 

5 

Radar  display:  Inappropriate  use  of  displayed  data:  Mode  C 

4 

5 

Coordination 

4 

5 

Data  posting:  Computer  entry 

3 

4 

Data  posting:  Flight  progress  strip 

2 

2 

Radar  display:  Misidentification:  Overlapping  data  blocks 

2 

2 

Unknown 

1 

1 
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TABLE  7.  SECONDARY  CAUSAL  FACTORS 


Secondary  Causal  Factor 

Number 

Percent 

Communication  error 

10 

42 

Radar  display:  Inappropriate  use  of  displayed  data 

8 

33 

Coordination 

5 

21 

Radar  display:  Misidentificatlon:  Climbed  aircraft  with  similar  call  sign 

1 

4 

TABLE  8.  SUMMARY  OF  ALL  CAUSAL  FACTORS 


Summary  of  Causal  Factors 

Number 

Percent 

69 

81 

Communication  error 

25 

29 

Coordination 

9 

11 

5 

6 

Unknown 

1 

1 

TABLE  9.  CORRELATIONS  OF  SECTOR  VARIABLES  WITH  NUMBER 
OF  SECTOR  OEs  AND  OE  GROUP 


Correlations 

Variable 

Number  of  OEs 

OE  Group 

Number  of  major  airports 

N/A 

Number  of  VORTACs 

Amount  of  climbing/descending  traffic 

r=.33,  p=.029 

Degree  of  aircraft  mix 

r=-.28,  p=  .060 

Number  of  multiple  functions 

Number  of  required  procedures 

r=  .38,  p=  .010 

Effect  of  airline  hubbing 

Weather 

WMgiagtMItlikJ 

Number  of  complex  aircraft  routings 

NS 

Number  of  restricted  areas 

NS 

Requirements  for  sequencing  and  spacing 

NS 

Radio  frequency  congestion 

r=.49,p=.001 

Average  traffic  volume 

Total  16CF  score  (sum) 

r=  .38,  p=  .010 

Average  density 

NS 

Average  complexity 

r=.42,  p=.004 
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TABLE  10.  ANOVA  TESTS  FOR  DIFFERENCES  BETWEEN 
SECTOR  CHARACTERISTICS 


Variable 

Zero 

Low 

mSmM 

F 

Amount  of  climbing/descending  traffic 

5.3 

5.6 

6.6 

2.16 

Degree  of  aircraft  mix 

5.5 

4.0 

3.8 

Number  of  required  procedures 

4.5 

5.1 

6.0 

.074 

Weather 

4.7 

5.0 

6.1 

.032 

Radio  frequency  congestion 

4.1 

5.2 

6.1 

Average  traffic  volume 

4.7 

5.5 

6.0 

.087 

Total  16CF  score  (sum) 

70.3 

75.7 

3.13 

.054 

Average  complexity 

269.3 

343.8 

5.45 

.008 

TABLE  11.  CORRELATIONS  OF  OE  CHARACTERISTICS  WITH  NUMBER 
OF  SECTOR  OES  AND  OE  FREQUENCY  GROUP 


Correlations 

Variable 

liHiMil 

Number  of  aircraft  in  the  sector 

■bei&essh 

Ts  =  .20,  p  =  .067 

Complexity 

r=.28,  p=.011 

Workload  index 

r=.24,  p=.027 

Tf  =  .21,  p=  .052 

Vertical  separation 

r=.20,  p=.073 

NS 

Position  combined 

N/A 

ts  =  -.29,  p  =  .007 

TABLE  12.  REGRESSION  ANALYSIS  OF  OE  DATA 


REGRESSION  STATISTICS 


MULTIPLE  R  .58 

MULTIPLE  R2  .34 

ADJUSTED  R2  .31 

SE  OF  ESTIMATE  1.73 


ANALYSIS  OF  VARIANCE 


REGRESSION 

RESIDUAL 

SS 

64.77 

125.67 

df 

2 

42 

MS 

32.39 

2.99 

F  RATIO 

10.82 

p=.000 

VARIABLES  IN  THE  EQUATION 

VARIABLE  b 

SE 

b 

t  D  (2  TAIU 

FREQ.  CONG. 

.76 

.17 

.56 

4*.37  .000 

REST.  AREAS 

.34 

.14 

.32 

2.47  .017 

CONSTANT 

64.59 
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TABLE  13.  RELATIONSHIP  OF  FINDINGS  TO  LITERATURE  REVIEW 


Source _ Factors  or  Issues _ Relevant  Factors _ 

Arad  {1 964)  Conflicts  related  to  rules  of  separation,  average  Correlations  with  OE  rate: 

traffic  speed,  number  of  aircraft  under  control,  T raffic  volume  higher, 
sector  size,  and  flow  organization.  Complex  routings  higher, 

_ Sectors  smaller  in  high  OE  sectors. 

Arad,  et  al.  Routine  load  on  controller  affected  by  placement  No  relevant  findings 

(1964) _  of  sector  boundaries  with  respect  to  traffic  flow. _ 

Siddiqee  (1973)  Conflicts  in  en  route  airspace  occur  due  to  a  loss  SA  errors  had  more  horizontal 

of  horizontal  separation  between  aircraft  at  the  separation  than  no-SA  errors. 

_ _ same  altitudes. _ _ 

Schmidt  (1976)  Conflicts  predicted  by  traffic  flow  rate,  separation  Correlations  with  OE  rate: 

standards,  route  geometry,  aircraft  speed,  aircraft  T raffic  volume, 
flow  rate,  angle  of  airway  intersection,  number  of  Complex  routings, 
_ flight  levels,  and  amount  of  transitioning  traffic.  Climbing/descending  traffic. _ 


Couluris  and 
Schmidt  (1973) 

Cost  of  sectorization  is  additional  workload 
(coordination)  imposed  by  placement  of  sector 
boundaries. 

Correlations  with  OE  rate: 
(Coordination  NS). 

Coordination  problems  cited  in 

11%  of  OEs. 

Empson  (1987) 
Langan-Fox  and 
Empson  (1985) 

Controller  workload  is  related  to  airspace 
structure,  procedural  demands,  traffic  type,  and 
control  over  task  presentation  rate. 

Correlations  with  OE  rate: 

Complex  routings. 

Required  procedures. 

Aircraft  mix. 

Kinney,  et  al. 
(1977) 

OEs  occur  under  low  to  moderate  workload  and 
moderate  complexity.  In  en  route  centers,  95%  of 
errors  attributed  to  attention,  judgment,  or 
communications.  Most  errors  occur  in  level  flight. 

OEs  occur  with  above  average 
traffic  load.  Most  OEs  rated  as 
moderately  complex. 

Schroeder  (1982) 

Most  errors  occur  under  light  or  moderate 
workload.  Other  aspects  of  the  situation  [sector 
factors?]  apart  from  traffic  volume  determine 
workload.  Coordination  a  direct  or  contributing 
factor  in  many  errors. 

OEs  occur  with  above  average 
traffic  load.  Most  OEs  rated  as 
moderately  complex.  Coordination 
not  correlated  with  error  count  but 
cited  as  cause  in  1 1  %  of  errors. 

Stager  and 
Hameluck 
(1990),  Stager,  et 
al.  (1989) 

Definitions  of  direct  and  contributing  causes.  OEs 
occur  under  low  to  moderate  workload.  Causes 
are  attention,  judgment,  and  communication 
problems. 

OEs  occur  with  above  average 
traffic  load.  Most  OEs  rated  as 
moderately  complex. 

Redding  (1992) 

Failure  to  maintain  SA  cause  of  most  errors  under 
moderate  traffic  load.  Communication, 
coordination,  and  misuse  of  radar  data  account  for 
most  errors. 

High  error  sectors  also  had  many 
no-SA  errors. 

Primary  causes  of  OEs:  29% 
communication;  81  %  problems 
with  radar  display. 

Schroeder  and 
Nye(1993) 

OEs  occur  under  average  or  lower  traffic 
complexity.  Problems  with  radar  display, 
communication,  coordination,  and  data  posting 
most  frequent  causes. 

OEs  occur  with  above  average 
traffic  load.  Most  OEs  rated  as 
moderately  complex.  Primary 
causes  of  OEs:  29% 
communication;  81  %  problems 
with  radar  display 

Rodgers  and  Nye 
(1993) 

Most  OEs  occur  with  one  aircraft  in  level  flight  and 
another  descending  or  ascending.  Most  moderate 
errors  are  between  aircraft  in  level  flight. 

Horizontal,  not  vertical,  separation  varies  with 
severity.  Higher  horizontal  separation  for  SA  OEs. 

SA  errors  had  more  horizontal 
separation  than  no-SA  errors. 

TABLE  13.  RELATIONSHIP  OF  FINDINGS  TO  LITERATURE  REVIEW  (CONTINUED) 


Source 

Factors  or  Issues 

Relevant  Factore 

Fowler  (1880) 

Sector  complexity  effected  by  problems  with 
coordination,  procedures,  LOAs,  and  weather. 

Correlations  with  OE  rate; 

Complex  routings 

Weather  problems  higher  In  high 

OE  sectors. 

Buckley,  et  al¬ 
ii  883) 

Sector  geometry  and  traffic  density  Interact  to 
affect  controller  performance. 

Correlations  with  OE  rate; 

Traffic  volume. 

Complex  routings. 
Climbing/descending  traffic 

Stein  (1886) 

Controller  workload  related  to  clustering  of  aircraft 
in  a  small  amount  of  airspace,  number  of  hand- 
offs  outbound/inbound,  and  total  number  of  flights 
handled. 

Correlations  with  OE  rate; 

Traffic  volume 

High  error  sectors  were  sniallci 

Qrossberg  (1888) 

Sector  complexity  factors  Include  control 
adjustments  to  merge  and  space  aircraft,  climbing 
and  descending  aircraft  flight  paths,  mixture  of 
aircraft  types,  frequent  coordination,  and  heavy 
traffic 

Correlations  with  OE  rate; 
(Sequenclng/Spaclng  NS) 
Cllmbing/descending  traffic, 

(Traffic  mix  NS), 

Traffic  volume. 

Mogford,  et  al¬ 
ii  883) 

Moyford,  et  al. 
(1093) 

ATC  complexity  factors  that  may  affect  controller 
workload  (and  OEs): 

1 .  Number  of  climbing  and  descending  aircraft. 

2.  Degree  of  aircraft  rrilx 

3.  Number  of  Intersecting  flight  paths 

4.  Number  of  multiple  functions  controller  must 
perform. 

6.  Number  of  required  procedures  controller 
must  perform. 

6.  Number  of  military  flights. 

7.  Frequency  of  contacts  (coordination)  or 
interface  with  other  entities. 

8.  Extent  to  which  controller  Is  affected  by  airline 
hubbing. 

8.  Extent  to  which  controller  is  affected  by 
weather. 

10.  Number  of  complex  aircraft  routings. 

1 1 .  Extent  to  which  controller  Is  affected  by 
restricted  areas,  warning  areas,  and  military 
operating  areas. 

12.  Size  of  sector  airspace. 

13.  Requirement  for  longitudinal  sequencing  and 
spacing. 

14.  Adequacy  and  reliability  of  radio  and  radar 

Correlations  with  OE  rate; 

1 .  Climbing/descending  tiuflic, 

2.  (Traffic  mix  NS). 

3.  (Intersections  NS), 

4.  Multiple  functions, 

6.  Procedures, 

6.  (Military  traffic  NS), 

7.  (Coordination  NS), 

8  Hubbing, 

8.  Weather, 

10.  Complex  routings, 

11.  Restricted  areas, 

12.  (Size  NS), 

13.  (Sequencing/spacing  NS), 

14.  (Radio/radar  coverage  NS), 

16.  Frequency  congestion, 

16.  Traffic  volume. 

High  OE  sectors  have  more: 
Frequency  congestion. 

Problem  weather. 

Total  complexity. 

Sectors  smaller  in  high  OE  sectors. 

coverage. 

16.  Amount  of  radio  frequency  congestion. 

16.  Average  amount  of  traffic. 

Rodgers  and 
Manning  (1986) 

OE  time  period  shows  increases  In  sector  transit 
time,  handoff  acceptance  latency,  vertical 
separation,  and  aircraft  density  and  decrease  in 
number  of  handoffs  accepted. 

Aircraft  volume  higher  at  time  of 

OE. 

TABLE  14.  COMPARISON  OF  OE  CAUSES 


Causal  Factor 

1985  to  1988  Data 

1992  to  1995  Data 

57% 

81% 

Communication 

30% 

29% 

Coordination 

30% 

11% 

Data  Posting 

20% 

6% 

Reiief  Briefing 

4% 

NA 
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APPENDIX  A: 


PEARSON  CORRELATION  MATRIX  OF  SECTOR  VARIABLES 


-  ”  Correlation  Coefficients  -  - 


AVG_DEN_ 


CLDC 


COOR 


CUVOL 


AVG„COM. 


AVG_DEN 


CLDC 


COOR 


CUVOL 


FPAPCT 


FREQ 


FUNC 


HUB 


INTR 


JET 


AVG_COM. 

1.0000 
{  45) 

P=  . 

.2762 
(  45) 

P=  .066 

.5744 
(  45) 

P=  .000 


.2129 
(  45) 
P=  .160 

-.2830 
{  45) 
P=  .060 

.1908 
{  45) 
P=  .209 

.5430 
{  45) 
P=  .000 

.5162 
(  45) 
P=  .000 

.5288 
(  45) 
P=  .000 

.3572 
(  45) 
P=  .016 

-.2158 
{  45) 
P=  .155 


.2762 
{  45) 

P=  .066 

1.0000 
{  45) 

P=  . 

-.0895 
{  45) 

P=  .559 

.1481 
(  45) 

P=  .332 

.3294 
(  45) 

P=  .027 

.1156 
(  45) 

P=::  .449 

.2178 
{  45) 

P:=  .151 

.2229 
(  45) 

P=  .141 

-.0092 
(  45) 

P=  .952 

.2597 
(  45) 

P=  .085 

.3615 
(  45) 

P=  .015 


,5744 
(  45) 

P=  .000 

-.0895 
(  45) 

P=  .559 

1.0000 
(  45) 

P=  . 

.3129 
(  45) 

P=  .036 

-.6169 
{  45) 

P=  .000 

.2218 
{  45) 

P=  .143 

.2537 
(  45) 

P=  .093 

.6369 
(  45) 

P=  .000 

.5560 
(  45) 

P=  .000 

.1920 
(  45) 

P=  .206 

-.5612 
(  45) 

P=  .000 


.2129 
(  45) 

P=  .160 

.1481 
(  45) 

P=  .332 

.3129 
{  45) 

P=  .036 

1.0000 
{  45) 

P=  . 

-.1467 
(  45) 

P=  .336 

-.0196 
(  45) 

P=  .898 

.3168 
{  45) 

P=  .034 

.5070 
(  45) 

P=  .000 

-.0005 
(  45) 

P=  .997 

.3394 
(  45) 

P=  .023 

-.1441 
{  45) 

P=  .345 


-.2830 
{  45) 
P=  .060 

,3294 
{  45) 
P=  .027 

-.6169 
{  45) 
P=  .000 


-.1467 
(  45) 

P=  .336 

1.0000 
(  45) 

P=  . 

.0175 
{  45) 

P=  .909 

-.0954 
{  45) 

P=  .533 

-.2088 
(  45) 

P=  .169 

-.1656 
(  45) 

P=  .277 

.0142 
{  45) 

P=  .926 

.7146 
(  45) 

P=  .000 


FPAPCT 


.1908 
{  45) 

P=  .209 

.1156 
(  45) 

P=  .449 

.2218 
(  45) 

P=  .143 

-.0196 
(  45) 

P=  .898 

.0175 
(  45) 

P=  .909 

1.0000 
{  45) 

P=  . 

-.0245 
(  45) 

P=  .873 

,1233 
(  45) 

P=  .420 

.1217 
(  45) 

P=  .426 

.0207 
{  45) 

P=  .892 

-.0935 
(  45) 

P=  .541 


(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A1 


Correlation  Coefficients 


AVG_COM_ 

AVG_DEN_ 

CLDC 

COOR 

CUVOL 

FPAPCT 

MIL 

-.0172 

.0710 

.0260 

.2505 

-.1182 

.1326 

(  45) 

( 

45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .911 

p= 

.643 

P=  .865 

P=  .097 

P=  .439 

P=  .385 

MIX 

-.0332 

— 

.2633 

.4821 

-.0702 

-.3913 

.2453 

(  45) 

{ 

45) 

{  45) 

{  45) 

(  45) 

(  45) 

P=  .828 

p= 

.081 

P=  .001 

P=  .647 

P=  .008 

P=  .104 

NOINTSX 

.0947 

.0768 

.2984 

.2602 

-.2963 

.2063 

(  45) 

( 

45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .536 

p= 

.616 

P=:  .046 

P=  .084 

P=  .048 

P=  .174 

OTHRTS 

-.0839 

.4267 

-.4620 

.1087 

.7272 

-.1144 

{  45) 

( 

45) 

(  45) 

(  45) 

(  45) 

{  45) 

P=  .584 

p= 

.003 

P=  .001 

P=  .477 

P=  .000 

P=  .454 

PROC 

.5239 

.1773 

.6650 

.5227 

-.2464 

.1099 

(  45) 

{ 

45) 

{  45) 

(  45) 

(  45) 

(  45) 

P=  .000 

p= 

.244 

P=  .000 

P=  .000 

P=  .103 

P=  .472 

RAD 

.2989 

.2839 

.0556 

-.0797 

.0588 

-.1347 

(  45) 

( 

45) 

(  45) 

(  45) 

{  45) 

(  45) 

P=  .046 

p= 

.059 

P=  .717 

P=  .603 

P=  .701 

P=  .378 

REST 

.0476 

.0267 

.2230 

.0021 

-.2451 

.2471 

(  45) 

( 

45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .756 

p= 

.862 

P=  .141 

P=  .989 

P=  .105 

P=  .102 

RTNG 

.3271 

.1986 

.3279 

.3562 

-.0216 

-.2108 

{  45) 

( 

45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .028 

p= 

.191 

P=  .028 

P=  .016 

P=  .888 

P=  .165 

SHELF 

-.1221 

_ 

.0755 

.2558 

-.0442 

-.1605 

.2990 

(  45) 

( 

45) 

(  45) 

(  45) 

(  45) 

{  45) 

P=  .424 

p= 

.622 

P=  .090 

P=  .773 

P=  .292 

P=  .046 

SIZE 

.0080 

.3072 

-.3856 

-.1413 

.5719 

.1399 

{  45) 

( 

45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .959 

.  p= 

.040 

P=  .009 

P=  .354 

P=  .000 

P=  .359 

SQ_SP 

.4424 

.1495 

.3634 

.1680 

-.0320 

-.0826 

(  45) 

{ 

45) 

{  45) 

(  45) 

(  45) 

(  45) 

P=  .002 

p= 

.327 

P=  .014 

P=  .270 

P=  .835 

P=  .590 

(Coefficient  /  (Cases) 

/ 

2-tailed 

Significance) 

"  .  ”  is 

printed  if  a 

coefficient 

cannot  be 

computed 

A2 


Correlation  Coefficients 


AVG_COM_ 

AVG_DEN„ 

CLDC 

COOR 

CUVOL 

FPAPCT 

TOTALOE 

.4189 

.2108 

,3259 

1482 

-.1354 

.1073 

(  45) 

(  45) 

{ 

45) 

( 

45) 

{  45) 

(  45) 

P=  .004 

P=  .165 

p= 

.029 

p= 

.331 

P=  .375 

P=  .483 

TOTCOMX 

.6951 

.2521 

.7285 

,4835 

-.2623 

.1533 

(  45) 

(  45) 

( 

45) 

( 

45) 

(  45) 

(  45) 

P=  .000 

P=  .095 

p= 

.000 

p= 

.001 

P=  .082 

P=  .315 

VICTOR 

-.1124 

.4612 

.5220 

.0198 

.7778 

-.1142 

(  45) 

(  45) 

( 

45) 

( 

45) 

(  45) 

(  45) 

P=  .462 

P=  .001 

p= 

.000 

p= 

.898 

P=  .000 

P=  .455 

VOL 

.5943 

.2614 

.3938 

.1560 

-.0806 

-.0023 

(  45) 

{  45) 

( 

45) 

( 

45) 

{  45) 

(  '  45) 

P=  .000 

P=  .083 

p= 

.007 

p= 

.306 

P=  .599 

P=  .988 

VORTAC 

.0307 

.3663 

_ 

.0800 

_ , 

.0011 

.2872 

-.0075 

(  45) 

(  45) 

( 

45) 

( 

45) 

(  45) 

(  45) 

P=  .842 

P=  .013 

p= 

.602 

p= 

.994 

P=  .056 

P=:  .961 

WX 

.5641 

.1766 

.4149 

.3467 

-.2065 

.1607 

(  45) 

(  45) 

( 

45) 

{ 

45) 

{  45) 

(  45) 

P=  .000 

P=  .246 

p= 

.005 

p= 

.020 

P=  .174 

P=  .292 

(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


Correlation  Coefficients 


FREQ 

FUNC 

HUB 

INTR 

JET 

MIL 

AVG_COM_ 

.5430 

.5162 

.5288 

.3572 

-.2158 

,0172 

(  45) 

(  45) 

(  45) 

{  45) 

(  45) 

( 

45) 

P=  .000 

P=  .000 

P=  .000 

P=  .016 

P=  .155 

p= 

.911 

AVG_DEN_ 

.2178 

.2229 

-.0092 

.2597 

.3615 

,0710 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

( 

45) 

P=  .151 

P=  .141 

P=  .952 

P=  .085 

P=  .015 

p= 

.643 

CLDC 

.2537 

.6369 

.5560 

.1920 

-.5612 

.0260 

(  45) 

{  45) 

(  45) 

{  45) 

(  45) 

( 

45) 

P=  .093 

P=  .000 

P=  .000 

P=  .206 

P=  .000 

p= 

.865 

COOR 

.3168 

.5070 

-.0005 

.3394 

-.1441 

.2505 

(  45) 

(  45) 

{  45) 

(  45) 

{  45) 

{ 

45) 

P=  .034 

P=  .000 

P=  .997 

P=  .023 

P=  .345 

p= 

.097 

CUVOL 

-.0954 

-.2088 

-.1656 

.0142 

.7146 

-  ^ 

.1182 

(  45) 

{  45) 

(  45) 

{  45) 

(  45) 

( 

45) 

P=  .533 

P=  .169 

P=  .277 

P=  .926 

P=  .000 

P= 

.439 

FPAPCT 

-.0245 

.1233 

.1217 

.0207 

-.0935 

.1326 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

( 

45) 

P=  .873 

P=  .420 

P=  .426 

P=  .892 

P=  .541 

P= 

.385 

FREQ 

1.0000 

.5291 

.4819 

.3704 

.0315 

.0269 

(  45) 

{  45) 

(  45) 

{  45) 

(  45) 

( 

45) 

P=  . 

P=  .000 

P=  .001 

P=  .012 

P=  .837 

P= 

.861 

FUNC 

.5291 

1.0000 

.2319 

.5898 

-.2477 

.2592 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

{ 

45) 

P=  .000 

P=  . 

P=  .125 

P=  .000 

P=  .101 

P= 

.086 

HUB 

.4819 

.2319 

1.0000 

.0784 

-.0944 

-  , 

.2476 

(  45) 

(  45) 

{  45) 

(  45) 

(  45) 

( 

45) 

P=  .001 

P=  .125 

P=  . 

P=  .609 

P=  .537 

P= 

.101 

INTR 

.3704 

.5898 

.0784 

1.0000 

.1128 

.1158 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

( 

45) 

P=  .012 

P=  .000 

P=  .609 

P=  . 

P=  .461 

P= 

.449 

JET 

.0315 

-.2477 

-.0944 

.1128 

1.0000 

- 

.0992 

(  45) 

(  45) 

(  45) 

{  45) 

(  45) 

( 

45) 

P=  .837 

P=  .101 

P=  .537 

P=  .461 

P=  . 

P= 

.517 

(Coefficient  /  (Cases) 

/  2-tailed 

Significance) 

"  .  "is 

printed  if  a 

coefficient 

cannot  be 

computed 

A4 


Correlation  Coefficients 


MIL 


MIX 


NOINTSX 


OTHRTS 


PROC 


RAD 


REST 


RTNG 


SHELF 


SIZE 


SQ_SP 


FREQ 

.0269 
(  45) 

P=  .861 

-.4366 
(  45) 

P=  .003 

.0101 

(  45) 

P=  .947 

.1693 
(  45) 

P=  .266 

.5505 
(  45) 

P=  .000 

.2495 
{  45) 

P=  .098 

-.2176 
{  45) 

P=  .151 

.3763 
(  45) 

P=  .011 

-.2077 
(  45) 

P=  .171 

.1444 
(  45) 

P=  .344 

.4325 
{  45) 

P=  .003 


FUNC 

.2592 
(  45) 

P=  .086 

.0880 
(  45) 

P=  .565 

.2497 
(  45) 

P=  .098 

-.0369 
(  45) 

P=  .810 

.8828 
{  45) 

P=  .000 

-.0175 
(  45) 

P=  .909 

.1059 
(  45) 

P=  .489 

.5365 
{  45) 

P=  .000 

.0914 
(  45) 

P=  .550 

-.1416 
{  45) 

P=  .353 

.4313 
(  45) 

P=  .003 


HUB 

-.2476 
{  45) 

P=  .101 

.0710 
(  45) 

P=  .643 

-.0721 
{  45) 

P=  .638 

-.0791 
(  45) 

P=  .606 

.3623 
(  45) 

P=  .014 

.2669 
(  45) 

P=  ,076 

-.0110 
(  45) 

P=  .943 

.2158 
{  45) 

P=  .155 

-.0376 
(  45) 

P=  .806 

.1047 
(  45) 

P=  .494 

.4593 
(  45) 

P=  .002 


INTR 

.1158 
(  45) 

P=  .449 

-.1456 
(  45) 

P=  .340 

.0292 
(  45) 

P=  .849 

.2528 
(  45) 

P=  .094 

.5121 
(  45) 

P=  .000 

-.1016 
(  45) 

P=  .507 

.0000 
(  45) 

P=1.000 

.3050 
(  45) 

P=  .042 

-.1514 
{  45) 

P=  .321 

.0096 
(  45) 

P=  .950 

.3193 
(  45) 

P=  .033 


JET 

-.0992 
(  45) 

P=  .517 

-.5476 
(  45) 

P=  .000 

-.4776 
(  45) 

P=  .001 

.8466 
(  45) 

P=  .000 

-.2100 
{  45) 

P=  .166 


.0277 
{  45) 

P=  .857 

-.2146 
(  45) 

P=  .157 

-.0112 
(  45) 

P=  .942 

-.3105 
{  45) 

P=  .038 

.6547 
(  45) 

P=  .000 

-.1510 
(  45) 

P=  .322 


MIL 

1.0000 
(  45) 

P=  . 

.0445 
(  45) 

P=  .772 

.1858 
{  45) 

P=  .222 

-.1221 
(  45) 

P=  .424 

.1406 
(  45) 

P=  .357 

-.4327 
(  45) 

P=  .003 

.5929 
(  45) 

P=  .000 

.1385 
{  45) 

P=  .364 

.2802 
{  45) 

P=  .062 

.0560 
(  45) 

P=  .715 

-.1842 
{  45) 

P=  .226 


(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A5 


-  -  Correlation  Coefficients  -  - 


FREQ 

FUNC 

HUB 

INTR 

JET 

MIL 

TOTALOE 

,4943 

.3280 

.3453 

.1375 

.0374 

.0418 

( 

45) 

( 

45) 

{  45) 

{  45) 

( 

45) 

(  45) 

p= 

.001 

p= 

.028 

P=  .020 

P=  .368 

p= 

.807 

P=  .785 

TOTCOMX 

.6542 

.8331 

.6129 

.5236 

-  , 

.1994 

.2320 

( 

45) 

( 

45) 

(  45) 

(  45) 

( 

45) 

{  45) 

p= 

.000 

p= 

.000 

P=  .000 

P=  .000 

p= 

.189 

P=  .125 

VICTOR 

.1622 

—  ^ 

.0895 

-.0551 

.2845 

.8879 

-.0951 

( 

45) 

( 

45) 

{  45) 

{  45) 

( 

45) 

(  45) 

p= 

.287 

p= 

.559 

P=  .719 

P=  .058 

p= 

.000 

P=  .534 

VOL 

.6815 

.5107 

.6355 

.3882 

.1451 

-.1795 

( 

45) 

( 

45) 

(  45) 

(  45) 

( 

45) 

(  45) 

p= 

.000 

p= 

.000 

P=  .000 

P=  .008 

p= 

.342 

P=  .238 

VORTAC 

— 

.1057 

.0149 

-.0878 

.2792 

.2162 

-.2835 

( 

45) 

( 

45) 

(  45) 

(  45) 

( 

45) 

(  45) 

p= 

.490 

p= 

.923 

P=  .566 

P=  .063 

p= 

.154 

P=  .059 

wx 

.6244 

.4854 

.3879 

.3248 

-  ^ 

.0322 

.2020 

( 

45) 

( 

45) 

(  45) 

(  45) 

( 

45) 

(  45) 

p= 

.000 

p= 

.001 

P=  .008 

P=  .030 

p= 

.834 

P=  .183 

{Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A6 


Correlation  Coefficients 


AVG^COM. 


AVG^DEN 


CLDC 


COOR 


CUVOL 


FPAPCT 


FREQ 


FUNC 


HUB 


INTR 


JET 


MIX 

-.0332 
(  45) 

P=  .828 

-.2633 
(  45) 

P=  .081 

.4821 
(  45) 

P=  .001 

-.0702 
{  45) 

P=  .647 

-.3913 
(  45) 

P=  .008 

.2453 
(  45) 

P=  .104 

-.4366 
(  45) 

P=  .003 

.0880 
{  45) 

P=  .565 

.0710 
(  45) 

P=:  .643 

-.1456 
{  45) 

P=  .340 

-.5476 
(  45) 

P=  .000 


NOINTSX 

.0947 
(  45) 

P=  .536 

.0768 
(  45) 

P=  .616 

.2984 
{  45) 

P=  .046 

.2602 
(  45) 

P=  .084 

-.2963 
(  45) 

P=  .048 

.2063 
(  45) 

P=  .174 

.0101 

(  45) 

P=  .947 

.2497 
(  45) 

P=  .098 

-.0721 
{  45) 

P=  .638 

.0292 
(  45) 

P=  .849 

-.4776 
{  45) 

P=  .001 


OTHRTS 


-.0839 
(  45) 
P=  .584 

.4267 
(  45) 
P=  .003 

-.4620 
(  45) 
P=  .001 

.1087 
{  45) 
P=  .477 

.7272 
(  45) 
P=  .000 

-.1144 
{  45) 
P=  .454 

.1693 
(  45) 
P=  .266 

-.0369 
(  45) 
P=  .810 

-.0791 
(  45) 
P=  .606 

.2528 
(  45) 
P=  .094 

.8466 
(  45) 
P=  ,000 


PROC 


.5239 
(  45) 
P=  .000 


.1773 
{  45) 

P=  .244 

.6650 
(  45) 

P=  .000 

.5227 
(  45) 

P=  .000 

-.2464 
(  45) 

P=  .103 

.1099 
{  45) 

P=  .472 

.5505 
(  45) 

P=  .000 

.8828 
(  45) 

P=  .000 

.3623 
(  45) 

P=  .014 

.5121 
(  45) 

P=  .000 

-.2100 
(  45) 

P=  .166 


RAD 


.2989 
(  45) 

P=  .046 

.2839 
(  45) 

P=  .059 

.0556 
{  45) 

P=  .717 

-.0797 
(  45) 

P=  .603 

.0588 
(  45) 

P=  .701 

-.1347 
(  45) 

P=  .378 

.2495 
(  45) 

P-  .098 

-.0175 
(  45) 

P=  .909 

.2669 
{  45) 

P=  .076 

-.1016 
(  45) 

P=  .507 

.0277 
{  45) 

P=  .857 


REST 

.0476 
(  45) 

P=  .756 

.0267 
{  45) 

P=  .862 

.2230 
(  45) 

P=  .141 

.0021 

{  45) 

P=  .989 

-.2451 
(  45) 

P=  .105 

.2471 
(  45) 

P=  .102 

-.2176 
(  45) 

P=  .151 

.1059 
(  45) 

P=  .489 

-.0110 
{  45) 

P=  .943 

.0000 

(  45) 

P-1.000 

-.2146 
(  45) 

P=  .157 


(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A7 


Correlation  Coefficients 


MIX 

NOINTSX 

OTHRTS 

PROC 

RAD 

REST 

MIL 

.0445 

.1858 

-.1221 

.1406 

-.4327 

.5929 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

{  45) 

P=  .772 

P=  .222 

P=  .424 

P=  .357 

P=  .003 

P=  .000 

MIX 

1.0000 

.4975 

-.6258 

.0499 

-.2040 

.2625 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  . 

P=  .001 

P=  .000 

P=  .745 

P=  .179 

P=  .082 

NOINTSX 

.4975 

1.0000 

-.3431 

.1686 

-.1741 

.2560 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .001 

P=  . 

P=  .021 

P=  .268 

P=  .253 

P=  .090 

OTHRTS 

-.6258 

-.3431 

1.0000 

.0495 

.1064 

-.4039 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .000 

P=  .021 

P=  . 

P=  .747 

P=  .487 

P=  .006 

PROC 

.0499 

.1686 

.0495 

1.0000 

.0028 

.0180 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .745 

P-  .268 

P=  .747 

P=  . 

P=  .986 

P=  .907 

RAD 

-.2040 

-.1741 

.1064 

.0028 

1.0000 

-.4421 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .179 

P=  .253 

P=  .487 

P=  .986 

P=  . 

P=  .002 

REST 

.2625 

.2560 

-.4039 

.0180 

-.4421 

1.0000 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .082 

P=  .090 

P=  .006 

P=  .907 

P=  .002 

P=  . 

RTNG 

-.1618 

.1139 

.2106 

.5460 

.2088 

-.1153 

(  45) 

(  45) 

(  45) 

(  45) 

{  45) 

(  45) 

P=  .288 

P=  .456 

P=  .165 

P=  .000 

P=  .169 

P=  .451 

SHELF 

.5947 

.5237 

-.3032 

.0297 

-.4060 

.2873 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .000 

P=  .000 

P=  .043 

P=  .846 

P=  .006 

P=  .056 

SIZE 

-.3966 

-.3113 

.6077 

-.1728 

.0071 

-.1132 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .007 

P=  .037 

P=  .000 

P=  .256 

P-  .963 

P=  .459 

SQ^SP 

-.1417 

-.0713 

.0907 

.5229 

.3489 

-.2600 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

(  45) 

P=  .353 

P=  .641 

P=  .554 

P=  .000 

P=  .019 

P=  .085 

(Coefficient  /  (Cases) 

/  2-tailed 

Significance) 

"  .  "  is 

printed  if  a 

coefficient 

cannot  be 

computed 

A8 


-  -  Correlation  Coefficients  -  - 


MIX  NOINTSX  OTHRTS  PROC  RAD  REST 


TOTALOE 

-.2827 

-.1273 

.0004 

,3818 

-.0129 

1946 

(  45) 

(  45) 

( 

45) 

{ 

45) 

{  45) 

( 

45) 

P=  .060 

P=  .405 

P= 

.998 

p= 

.qio 

P=  .933 

p= 

.200 

TOTCOMX 

.0646 

.2160 

—  ^ 

.0137 

.8419 

.1291 

1390 

(  45) 

(  45) 

( 

45) 

( 

45) 

(  45) 

{ 

45) 

P=  .673 

P=  .154 

P= 

.929 

p= 

.000 

P=  .398 

p= 

.362 

VICTOR 

-.6223 

-.4634 

.9054 

—  _ 

.0583 

.1542 

—  _ 

3226 

(  45) 

(  45) 

( 

45) 

( 

45) 

(  45) 

( 

45) 

P=  .000 

P=  .001 

P= 

.000 

p= 

.704 

P=  .312 

p= 

.031 

VOL 

-.1948 

-.0138 

.2226 

.5653 

.2156 

— . 

1715 

(  45) 

(  45) 

( 

45) 

( 

45) 

(  45) 

( 

45) 

P=  .200 

P=  .928 

P= 

.142 

p= 

.000 

P=:  .155 

p= 

.260 

VORTAC 

.1241 

.3588 

.2923 

— , 

.0470 

.2068 

_  ^ 

2130 

(  45) 

{  45) 

{ 

45) 

{ 

45) 

(  45) 

( 

45) 

P=  .417 

P=  .015 

P= 

.051 

p= 

.759 

P=  .173 

p= 

.160 

WX 

-.1692 

.1482 

.1434 

.5197 

.0904 

1058 

(  45) 

(  45) 

( 

45) 

( 

45) 

(  45) 

( 

45) 

P=  .266 

P=  .331 

P= 

.347 

p= 

.000 

P=  .555 

p= 

.489 

(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A9 


Correlation  Coefficients 


RTNG 


SHELF  SIZE  SQ_SP 


TOTALOE  TOTCOMX 


AVG_COM_ 

( 

.3271 

45) 

-.1221 
(  45) 

( 

,0080 

45) 

( 

.4424 

45) 

( 

.4189 

45) 

( 

.6951 

45) 

p= 

.028 

P=  .424 

p= 

.959 

p= 

.002 

p= 

.004 

p= 

.000 

AVG_DEN_ 

{ 

.1986 

45) 

-.0755 
(  45) 

( 

,3072 

45) 

( 

.1495 

45) 

( 

.2108 

45) 

( 

.2521 

45) 

p= 

.191 

P=  .622 

p= 

.040 

p= 

.327 

p= 

.165 

p= 

.095 

CLDC 

( 

.3279 

45) 

.2558 
(  45) 

( 

.3856 

45) 

( 

.3634 

45) 

( 

.3259 

45) 

{ 

.7285 

45) 

p= 

.028 

P=  .090 

p= 

.009 

p= 

.014 

p= 

.029 

p= 

.000 

COOR 

( 

.3562 

45) 

-.0442 
(  45) 

( 

.1413 

45) 

( 

.1680 

45) 

( 

.1482 

45) 

{ 

.4835 

45) 

p= 

.016 

P=  .773 

p= 

.354 

p= 

.270 

p= 

.331 

p= 

.001 

CUVOL 

.0216 

-.1605 

.5719 

.0320 

- 

.1354 

- 

.2623 

{ 

45) 

(  45) 

( 

45) 

( 

45) 

( 

45) 

( 

45) 

p= 

.888 

P=  .292 

p= 

.000 

p= 

.835 

p= 

.375 

p= 

.082 

FPAPCT 

{ 

.2108 

45) 

.2990 
(  45) 

( 

.1399 

45) 

( 

.0826 

45) 

( 

.1073 

45) 

( 

.1533 

45) 

p= 

.165 

P=  .046 

p= 

.359 

p= 

.590 

p= 

.483 

p= 

.315 

FREQ 

( 

.3763 

45) 

-.2077 
(  45) 

( 

.1444 

45) 

( 

.4325 

45) 

( 

.4943 

45) 

{ 

.6542 

45) 

p= 

.011 

P=  .171 

p= 

.344 

p= 

.003 

p= 

.001 

p= 

.000 

FUNC 

( 

.5365 

45) 

.0914 
(  45) 

( 

.1416 

45) 

( 

.4313 

45) 

( 

.3280 

45) 

( 

.8331 

45) 

p= 

.000 

P=  .550 

p= 

.353 

p= 

.003 

p= 

.028 

p= 

.000 

HUB 

{ 

.2158 

45) 

-.0376 
(  45) 

( 

.1047 

45) 

{ 

.4593 

45) 

( 

.3453 

45) 

( 

.6129 

45) 

p= 

.155 

P=  .806 

p= 

.494 

p= 

.002 

p= 

.020 

p= 

.000 

INTR 

( 

.3050 

45) 

-.1514 
(  45) 

( 

.0096 

45) 

( 

.3193 

45) 

( 

.1375 

45) 

( 

.5236 

45) 

p= 

.042 

P=  .321 

p= 

.950 

p= 

.033 

p= 

.368 

p= 

.000 

JET 

( 

.0112 

45) 

-.3105 
(  45) 

( 

.6547 

45) 

( 

.1510 

45) 

( 

.0374 

45) 

{ 

.1994 

45) 

p= 

.942 

P=  .038 

p= 

.000 

p= 

.322 

p= 

.807 

p= 

.189 

(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


AlO 


Correlation  Coefficients 


MIL 


MIX 


NOINTSX 


GTHRTS 


PROC 


RAD 


REST 


RTNG 


SHELF 


SIZE 


SQ_SP 


RTNG 

.1385 
{  45) 

P=  .364 

-.1618 
(  45) 

P=  .288 

.1139 
(  45) 

P=  .456 

.2106 
(  45) 

P=  .165 

.5460 
(  45) 

P=  .000 

.2088 
(  45) 

P=  .169 

-.1153 
{  45) 

P=  .451 

1.0000 

(  45) 

P=  . 

-.0880 
(  45) 

P=  .566 

-.1130 
(  45) 

P=  .460 

.5172 
(  45) 

P=  .000 


SHELF 

.2802 
(  45) 

P=  .062 

.5947 
{  45) 

P=  .000 

.5237 
(  45) 

P=  .000 

-.3032 
(  45) 

P=  .043 

.0297 
(  45) 

P=  .846 

-.4060 
(  45) 

P=  .006 

.2873 
(  45) 

P=  .056 

-.0880 
(  45) 

P=  .566 

1.0000 

(  45) 

P=  . 

-.1619 
(  45) 

P=  .288 

-.3071 
{  45) 

P=  .040 


SIZE 

.0560 
(  45) 

P=  .715 

-.3966 
(  45) 

P=  .007 

-.3113 
(  45) 

P=  .037 

.6077 
(  45) 

P=  .000 

-.1728 
(  45) 

P=  .256 

.0071 
(  45) 

P=  .963 

-.1132 
(  45) 

P=  .459 

-.1130 
(  45) 

P=  .460 

-.1619 
(  45) 

P=  .288 

1.0000 

(  45) 

P=  . 

.0020 
(  45) 

P=  .990 


SQ_SP 

-.1842 
(  45) 

P=  .226 

-.1417 
{  45) 

P=  .353 

-.0713 
(  45) 

P=  .641 

.0907 
(  45) 

P=  .554 

.5229 
(  45) 

P=  .000 

.3489 
(  45) 

P=  .019 

-.2600 
(  45) 

P=  .085 

.5172 
(  45) 

P=  .000 

-.3071 
(  45) 

P=  .040 

.0020 

(  45) 

P=  .990 

1.0000 

(  45) 

P=  . 


TOTALOE 


.0418 
(  45) 

P=  .785 

-.2827 
{  45) 

P=  .060 

-.1273 
{  45) 

P=  .405 

-.0004 
(  45) 

P^  .998 

.3818 
(  45) 

P=  .010 

-.0129 
(  45) 

P=  .933 

.1946 
(  45) 

P=  .200 

.2363 
{  45) 

P=  .118 

-.0684 
{  45) 

P=  .655 

-.1221 
{  45) 

P=  .424 

.0929 
(  45) 

P=  .544 


TOTCOMX 

.2320 
{  45) 

P=  .125 

.0646 
(  45) 

P=  .673 

.2160 
(  45) 

P=  .154 

-.0137 
(  45) 

P=  .929 

.8419 
{  45) 

P=  .000 

.1291 
(  45) 

P=  .398 

.1390 
(  45) 

P=  .362 

.6004 
(  45) 

P=  .000 

.0283 
(  45) 

P=  .854 

.0108 
(  45) 

P=  .944 

.6180 
(  45) 

P=  .000 


(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


All 


Correlation  Coefficients 


TOTALOE 


TOTCOMX 


VICTOR 


VOL 


VORTAC 


WX 


RTNG 


.2363 
(  45) 
P=  .118 

.6004 
(  45) 
P=  .000 

.1317 
(  45) 
P=  .389 


.3808 
(  45) 
P=  .010 

.0899 
(  45) 
P=  .557 

.3781 
{  45) 
P=  .010 


SHELF 

-.0684 
(  45) 

P=  .655 

.0283 
(  45) 

P=  .854 

-.4125 
(  45) 

P=  .005 

-  .1708 
{  45) 

P=  .262 

.1623 
(  45) 

P=  .287 

.0182 
(  45) 

P=:  .905 


SIZE 

-.1221 
(  45) 

P=  .424 

.0108 
(  45) 

P=  .944 

.6531 
(  45) 

P=  .000 

.2043 
(  45) 

P=  .178 

.1994 
(  45) 

P=  .189 

.0843 
(  45) 

P=  .582 


SQ_SP 

.0929 
{  45) 

P=  .544 

.6180 
(  45) 

P=  .000 

.1144 
(  45) 

P=  .454 

.5330 
{  45) 

P=  .000 

.0718 
{  45) 

P=  .639 

.3606 
(  45) 

P=  .015 


TOTALOE 

1.0000 
(  45) 

P=  . 

.3821 
{  45) 

P=  .010 

-.0241 
(  45) 

P=  .875 

.3338 
(  45) 

P=  .025 

-.3050 
(  45) 

P=  .042 

.4493 
(  45) 

P=  .002 


TOTCOMX 

.3821 
(  45) 

P=  .010 

1.0000 
{  45) 

P=  . 

-.0382 
(  45) 

P=  .803 

.7066 
(  45) 

P=  .000 

.0157 
{  45) 

P=  .918 

.6685 
(  45) 

P=  .000 


(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A12 


Correlation  Coefficients 


AVG_COM. 


AVG_DEN 


CLDC 


COOR 


CUVOL 


FPAPCT 


FREQ 


FUNC 


HUB 


INTR 


JET 


VICTOR 

-.1124 
(  45) 
P=  .462 


.4612 
{  45) 

P=  .001 

-.5220 
(  45) 

P=  .000 

.0198 
(  45) 

P=  .898 

.7778 
(  45) 

P=  .000 

-.1142 
(  45) 

P=  .455 

.1622 
{  45) 

P=  .287 

-.0895 
(  45) 

P=  .559 

-.0551 
(  45) 

P=  .719 

.2845 
(  45) 

P=  .058 

.8879 
{  45) 

P=  .000 


VOL 

.5943 
{  45) 

P=  .000 

.2614 
(  45) 

P=  .083 

.3938 
{  45) 

P=  .007 

.1560 
(  45) 

P=  .306 

-.0806 
(  45) 

P=  .599 

-.0023 
{  45) 

P=  .988 

.6815 
(  45) 

P=  .000 

.5107 
(  45) 

P=  .000 

.6355 
{  45) 

P=  .000 

.3882 
(  45) 

P=  .008 

.1451 
(  45) 

P=  .342 


VORTAC 

.0307 
(  45) 
P=  .842 


.3663 
(  45) 

P=  .013 

-.0800 
(  45) 

P=  .602 

-.0011 
(  45) 

P=  .994 

.2872 
{  45) 

P=  .056 

-.0075 
(  45) 

P=  .961 

-.1057 
(  45) 

P=  .490 

.0149 
{  45) 

P=  .923 

-.0878 
(  45) 

P=  .566 

.2792 
(  45) 

P=  .063 

.2162 
{  45) 

P=  .154 


WX 

.5641 
(  45) 
P=  .000 

.1766 
{  45) 
P=  .246 

.4149 
(  45) 
P=  .005 

.3467 
(  45) 
P=  .020 

-.2065 
{  45) 
P=  .174 

.1607 
(  45) 
P=  .292 

.6244 
{  45) 
P=  .000 

.4854 
{  45) 
P=  .001 

.3879 
(  45) 
P=  .008 

.3248 
{  45) 
P=  .030 

-.0322 
(  45) 
P=  .834 


(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A13 


Correlation  Coefficients 


MIL 

MIX 

NOINTSX 

OTHRTS 

PROC 

RAD 

REST 

RTNG 

SHELF 

SIZE 

SQ„SP 


VICTOR 

-.0951 
{  45) 
P=  .534 

-.6223 
{  45) 
P=  .000 


-.4634 
{  45) 
P=  .001 


.9054 
{  45) 
P=  .000 


-.0583 
{  45) 
P=  .704 

.1542 
(  45) 
P=  .312 

-.3226 
(  45) 
P=  .031 

.1317 
(  45) 
P=  .389 

-.4125 
(  45) 
P=  .005 

.6531 
{  45) 
P=  .000 

.1144 
(  45) 
P=  .454 


VOL 

-.1795 
(  45) 

P=  .238 

-.1948 
{  45) 

P=  .200 

-.0138 
(  45) 

P=  .928 

.2226 
{  45) 

P=  .142 

.5653 
{  45) 

P=  .000 

.2156 
(  45) 

P=  .155 

-.1715 
(  45) 

P=  .260 

.3808 
(  45) 

P=  .010 

-.1708 
(  45) 

P=  .262 

.2043 
(  45) 

P=  .178 

.5330 
(  45) 

P=  .000 


VORTAC 

-.2835 
(  45) 

P=  .059 

.1241 
(  45) 

P=  .417 

.3588 
(  45) 

P=  .015 

.2923 
(  45) 

P=  .051 

-.0470 
(  45) 

P=  .759 

.2068 
{  45) 

P=  .173 

-.2130 
(  45) 

P=  .160 

.0899 
(  45) 

P=  .557 

.1623 
(  45) 

P=  .287 

.1994 
(  45) 

P=  .189 

.0718 
(  45) 

P=  .639 


WX 

.2020 

(  45) 

P=  .183 

-.1692 
(  45) 

P=  .266 

.1482 
(  45) 

P=  .331 

.1434 
{  45) 

P=  .347 

.5197 
(  45) 

P=  .000 

.0904 
{  45) 

P=  .555 

-.1058 
{  45) 

P=  .489 

.3781 
(  45) 

P=  .010 

.0182 
(  45) 

P=  .905 

.0843 
(  45) 

P=  .582 

.3606 
(  45) 

P=  .015 


(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A14 


Correlation  Coefficients 


TOTALOE 


TOTCOMX 


VICTOR 


VOL 


VORTAC 


WX 


VICTOR 

-.0241 
(  45) 

P=  .875 

-.0382 
(  45) 

P=  .803 

1.0000 

(  45) 

P=  . 

.2232 
(  45) 

P=  .141 

.2596 
(  45) 

P=  .085 

-.0013 
{  45) 

P=:  .993 


VOL 

.3338 
(  45) 

P=  .025 

.7066 
{  45) 

P=  .000 

.2232 
(  45) 

P=  .141 

1.0000 
(  45) 

P=  . 

.0238 
(  45) 

P=  .876 

.5155 
{  45) 

P=  .000 


VORTAC 

-.3050 
(  45) 

P=  .042 

.0157 
(  45) 

P=  .918 

.2596 
{  45) 

P=  .085 

.0238 
{  45) 

P=  .876 

1.0000 
(  45) 

P=  . 

-.0232 
(  45) 

P=  .880 


WX 

.4493 
(  45) 

P=  .002 

.6685 
(  45) 

P=  .000 

-.0013 
{  45) 

P=  .993 

.5155 
{  45) 

P=  .000 

-.0232 
(  45) 

P=  .880 

1.0000 
{  45) 

P=  . 


(Coefficient  /  (Cases)  /  2-tailed  Significance) 

"  .  "  is  printed  if  a  coefficient  cannot  be  computed 


A15 
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